1 Loading (meta)data and annotations

The following examples will use a real data set from an experiment in our lab. The raw data was processed using a mix of trimmomatic, biopieces, bowtie, samtools, and htseq. The final count tables were deposited into the ‘preprocessing/count_tables/’ tree. The resulting data structure was named ‘most_v0M1,’ named because it is comprised of count tables with 0 mismatches and 1 randomly-placed multi-match.

The annotation file was mgas_5005.gff.xz resides in ‘reference/gff/’.

The count tables and meta-data were loaded through the create_expt() function and the genome annotations were loaded with gff2df().

library(hpgltools)
data_file <- system.file("share/cdm_expt.rda", package = "hpgltools")
cdm <- new.env()
load(data_file, envir = cdm)
rm(data_file)

ls()
## [1] "cdm"         "old_options" "rmd_file"

Two variables should exist now: rmd_file in case I want to knitr this file, cdm which is a list including the data required to make an expressionset. Using this information, I can create an expressionset.

expt <- create_expt(count_dataframe = cdm$cdm_counts,
                    metadata = cdm$cdm_metadata,
                    gene_info = cdm$gene_info)
## Reading the sample metadata.
## The sample definitions comprises: 8 rows(samples) and 22 columns(metadata fields).
## Matched 1926 annotations and counts.
## Bringing together the count matrix and gene information.
## Some annotations were lost in merging, setting them to 'undefined'.
## Saving the expressionset to 'expt.rda'.
## The final expressionset has 1926 rows and 8 columns.
knitr::kable(head(expt$design))
sampleid type stage replicate mutantname media exptdate libdate batch condition readspassed ncrna xncrna remaining genome xgenome otherbacterial xother colors counts intercounts design file
HPGL0418 HPGL0418 WT LL 1 5448-1 CDMg 20130430 20140409 a wt_ll_cg 17425675 350440 2.01% 17075235 16061860 94.07% 356542 2.09% #E12C8C processed_data/count_tables/HPGL0418/HPGL0418_forward-trimmed-v1M1l20.count.xz unknown a null
HPGL0419 HPGL0419 WT LL 2 5448-2 CDMg 20130430 20140409 b wt_ll_cg 18106361 398984 2.20% 17707377 16619183 93.85% 392803 2.22% #E12C8C processed_data/count_tables/HPGL0419/HPGL0419_forward-trimmed-v1M1l20.count.xz unknown b null
HPGL0420 HPGL0420 mga LL 1 Mga-1 CDMg 20130430 20140409 a mga1_ll_cg 20499936 417440 2.04% 20082496 18062942 89.94% 416260 2.07% #BE5067 processed_data/count_tables/HPGL0420/HPGL0420_forward-trimmed-v1M1l20.count.xz unknown a null
HPGL0421 HPGL0421 mga LL 2 Mga-2 CDMg 20130430 20140409 b mga1_ll_cg 17216381 397021 2.31% 16819360 14795703 87.97% 376295 2.24% #BE5067 processed_data/count_tables/HPGL0421/HPGL0421_forward-trimmed-v1M1l20.count.xz unknown b null
HPGL0422 HPGL0422 WT LL 1 5448-1 CDMf 20130430 20140415 a wt_ll_cf 17112706 367791 2.15% 16744915 15114930 90.27% 367974 2.20% #8E7E40 processed_data/count_tables/HPGL0422/HPGL0422_forward-trimmed-v1M1l20.count.xz unknown a null
HPGL0423 HPGL0423 WT LL 2 5448-2 CDMf 20130430 20140415 b wt_ll_cf 18782729 395748 2.11% 18386981 17330239 94.25% 386443 2.10% #8E7E40 processed_data/count_tables/HPGL0423/HPGL0423_forward-trimmed-v1M1l20.count.xz unknown b null
summary(expt)
##                  Length Class         Mode     
## title             1     -none-        character
## notes             1     -none-        character
## initial_metadata 23     data.frame    list     
## expressionset     1     ExpressionSet S4       
## design           23     data.frame    list     
## conditions        8     factor        numeric  
## batches           8     factor        numeric  
## samplenames       8     -none-        character
## colors            8     -none-        character
## state             5     -none-        list     
## libsize           8     -none-        numeric

2 Introduction

hpgltools was written to make working with high-throughput data analyses easier. These analyses generally fall into a few stages:

  1. Data visualization and outlier/batch evaluation
  2. Differential expression analyses
  1. Visualization and export of these results
  1. Gene ontology/KEGG analyses
  1. Visualization and export of these results
  1. Genome visualizations
  1. With circos
  2. With genoplotR

Before any of these tasks may be performed, the data must be loaded into memory. hpgltools attempts to make this easier with create_expt() and subset_expt().

3 A little side-track

I want to use this vignette to explore the relationship between expressionSets and SummarizedExperiments. I use the former extensively, and have noticed but not paid strong attention to the latter.

tt <- sm(library(SummarizedExperiment))
test_expr <- expt[["expressionset"]]

## The following in theory converts to a SE, but is a bit unhelpful in its result.
## Convert expressionSet to summarizedExperiment
test_se <- SummarizedExperiment::makeSummarizedExperimentFromExpressionSet(test_expr)
## Holy crap long function name!
test_assay <- assay(test_se)
head(test_assay)
##               HPGL0418 HPGL0419 HPGL0420 HPGL0421 HPGL0422 HPGL0423 HPGL0424
## M5005_Spy0001     8783     8012    13625     8701     8208     6495     8427
## M5005_Spy0002     9489    10574    12314     8983     9204     7205     8550
## M5005_Spy0003      550      682     1107      727      477      495      819
## M5005_Spy0004     9844    12848    20996    13849     7021     8898    15319
## M5005_Spy0005     1045     1328     1356     1021      775      769     1031
## M5005_Spy0006    12271    12033    14812    10522     7100     9422    11375
##               HPGL0425
## M5005_Spy0001     7661
## M5005_Spy0002     7717
## M5005_Spy0003      786
## M5005_Spy0004    13244
## M5005_Spy0005      931
## M5005_Spy0006    11658
test_meta <- colData(test_se)
head(test_meta)
## DataFrame with 6 rows and 23 columns
##          sampleid     type    stage replicate mutantname    media  exptdate
##          <factor> <factor> <factor> <integer>   <factor> <factor> <integer>
## HPGL0418 HPGL0418      WT        LL         1     5448-1     CDMg  20130430
## HPGL0419 HPGL0419      WT        LL         2     5448-2     CDMg  20130430
## HPGL0420 HPGL0420      mga       LL         1     Mga-1      CDMg  20130430
## HPGL0421 HPGL0421      mga       LL         2     Mga-2      CDMg  20130430
## HPGL0422 HPGL0422      WT        LL         1     5448-1     CDMf  20130430
## HPGL0423 HPGL0423      WT        LL         2     5448-2     CDMf  20130430
##            libdate    batch  condition readspassed     ncrna   xncrna remaining
##          <integer> <factor>   <factor>   <integer> <integer> <factor> <integer>
## HPGL0418  20140409        a wt_ll_cg      17425675    350440    2.01%  17075235
## HPGL0419  20140409        b wt_ll_cg      18106361    398984    2.20%  17707377
## HPGL0420  20140409        a mga1_ll_cg    20499936    417440    2.04%  20082496
## HPGL0421  20140409        b mga1_ll_cg    17216381    397021    2.31%  16819360
## HPGL0422  20140415        a wt_ll_cf      17112706    367791    2.15%  16744915
## HPGL0423  20140415        b wt_ll_cf      18782729    395748    2.11%  18386981
##             genome  xgenome otherbacterial   xother  colors
##          <integer> <factor>      <integer> <factor>  <list>
## HPGL0418  16061860   94.07%         356542    2.09% #E12C8C
## HPGL0419  16619183   93.85%         392803    2.22% #E12C8C
## HPGL0420  18062942   89.94%         416260    2.07% #BE5067
## HPGL0421  14795703   87.97%         376295    2.24% #BE5067
## HPGL0422  15114930   90.27%         367974    2.20% #8E7E40
## HPGL0423  17330239   94.25%         386443    2.10% #8E7E40
##                          counts intercounts      design        file
##                     <character> <character> <character> <character>
## HPGL0418 processed_data/count..     unknown           a        null
## HPGL0419 processed_data/count..     unknown           b        null
## HPGL0420 processed_data/count..     unknown           a        null
## HPGL0421 processed_data/count..     unknown           b        null
## HPGL0422 processed_data/count..     unknown           a        null
## HPGL0423 processed_data/count..     unknown           b        null
test_gene_info <- rowData(test_se)
head(test_gene_info)
## DataFrame with 6 rows and 25 columns
##                  seqnames       start         end       width      strand
##               <character> <character> <character> <character> <character>
## M5005_Spy0001   NC_007297         202        1557        1356           +
## M5005_Spy0002   NC_007297        1712        2848        1137           +
## M5005_Spy0003   NC_007297        2923        3120         198           +
## M5005_Spy0004   NC_007297        3450        4565        1116           +
## M5005_Spy0005   NC_007297        4635        5204         570           +
## M5005_Spy0006   NC_007297        5207        8710        3504           +
##                    source        type       score       phase            ID
##               <character> <character> <character> <character>   <character>
## M5005_Spy0001      RefSeq        gene   undefined   undefined M5005_Spy0001
## M5005_Spy0002      RefSeq        gene   undefined   undefined M5005_Spy0002
## M5005_Spy0003      RefSeq        gene   undefined   undefined M5005_Spy0003
## M5005_Spy0004      RefSeq        gene   undefined   undefined M5005_Spy0004
## M5005_Spy0005      RefSeq        gene   undefined   undefined M5005_Spy0005
## M5005_Spy0006      RefSeq        gene   undefined   undefined M5005_Spy0006
##                       Dbxref Is_circular       gbkey      genome    mol_type
##                  <character> <character> <character> <character> <character>
## M5005_Spy0001 GeneID:3571011   undefined        Gene   undefined   undefined
## M5005_Spy0002 GeneID:3571012   undefined        Gene   undefined   undefined
## M5005_Spy0003 GeneID:3571013   undefined        Gene   undefined   undefined
## M5005_Spy0004 GeneID:3571014   undefined        Gene   undefined   undefined
## M5005_Spy0005 GeneID:3571015   undefined        Gene   undefined   undefined
## M5005_Spy0006 GeneID:3571016   undefined        Gene   undefined   undefined
##                    strain           Name          Note        gene
##               <character>    <character>   <character> <character>
## M5005_Spy0001   undefined           dnaA M5005_Spy0001        dnaA
## M5005_Spy0002   undefined           dnaN M5005_Spy0002        dnaN
## M5005_Spy0003   undefined M5005_Spy_0003 M5005_Spy0003   undefined
## M5005_Spy0004   undefined           ychF M5005_Spy0004        ychF
## M5005_Spy0005   undefined            pth M5005_Spy0005         pth
## M5005_Spy0006   undefined           trcF M5005_Spy0006        trcF
##                    locus_tag       Parent     product  protein_id transl_table
##                  <character>  <character> <character> <character>  <character>
## M5005_Spy0001 M5005_Spy_0001 character(0)   undefined   undefined    undefined
## M5005_Spy0002 M5005_Spy_0002 character(0)   undefined   undefined    undefined
## M5005_Spy0003 M5005_Spy_0003 character(0)   undefined   undefined    undefined
## M5005_Spy0004 M5005_Spy_0004 character(0)   undefined   undefined    undefined
## M5005_Spy0005 M5005_Spy_0005 character(0)   undefined   undefined    undefined
## M5005_Spy0006 M5005_Spy_0006 character(0)   undefined   undefined    undefined
##               gene_synonym
##                <character>
## M5005_Spy0001 character(0)
## M5005_Spy0002 character(0)
## M5005_Spy0003 character(0)
## M5005_Spy0004 character(0)
## M5005_Spy0005 character(0)
## M5005_Spy0006 character(0)
## Here is a stupider version of the above, but one which will not make my fingers sad.
exprs_to_se <- function(exset) {
  mtrx <- Biobase::exprs(exset)
  annot <- Biobase::fData(exset)
  meta <- Biobase::pData(exset)
  se <- SummarizedExperiment(assays = mtrx, colData = meta, rowData = annot)
  return(se)
}

## and back?
test_ex <- as(test_se, "ExpressionSet")
head(exprs(test_ex))
##               HPGL0418 HPGL0419 HPGL0420 HPGL0421 HPGL0422 HPGL0423 HPGL0424
## M5005_Spy0001     8783     8012    13625     8701     8208     6495     8427
## M5005_Spy0002     9489    10574    12314     8983     9204     7205     8550
## M5005_Spy0003      550      682     1107      727      477      495      819
## M5005_Spy0004     9844    12848    20996    13849     7021     8898    15319
## M5005_Spy0005     1045     1328     1356     1021      775      769     1031
## M5005_Spy0006    12271    12033    14812    10522     7100     9422    11375
##               HPGL0425
## M5005_Spy0001     7661
## M5005_Spy0002     7717
## M5005_Spy0003      786
## M5005_Spy0004    13244
## M5005_Spy0005      931
## M5005_Spy0006    11658
## YAY!

4 Working with expts

The data structure generated by create_expt() is a list containing the following slots:

  • initial_metadata: A backup of the metadata
  • original_expressionset: A backup of the raw counts
  • expressionset: The current count data
  • samples: A data frame of metadata used for subsets
  • design: The design of the experiment
  • definitions: Extended design information, these are probably redundant and should be pruned.
  • stages: The experimental stage
  • types: Cell types
  • conditions: Experimental condition
  • batches: Experimental batch
  • samplenames: Names of the samples
  • colors: Colors chosen for graphs and such
  • names: Bringing together the condition/batch
  • filtered: low-count filtering status of the counts
  • transform: transformation applied to the counts
  • norm: normalization applied to the counts
  • convert: cpm/rpkm/etc applied to the data
  • original_libsize: the library sizes before normalization
  • columns: A backup of the sample names

The primary reason I created the expt object was that I did not fully understand how expressionSets worked. From my perspective, the documentation was rather opaque and therefore I decided to create a simpler version of the expressionset. As I learned how to manipulate S4 classes more efficiently, I gradually began to realize that they are not so bad. Nonetheless, I found it nice to be able to keep some extra state information with my expressionsets, along with a backup copy in case of mistakes, and some easier ways to extract information like conditions/batches/colors/etc. Thus my *expt() functions grew into wrappers to call the native functions for manipulating expressionsets.

5 Raw metrics

With the above in mind, once we have the various metadata and count data loaded into memory, a most common next step is examine the data before molesting it:

raw_metrics <- sm(graph_metrics(expt, qq = TRUE, cis = NULL))

The function graph_metrics() performs all of the likely plots one might want. Some of which are not really appropriate for non-normalized data unless it is incredibly well behaved (after 30 years, I still want to spell behaved ‘behaived’, why is that?).

## View a raw library size plot
raw_metrics$libsize

## Or boxplot to see the data distribution
raw_metrics$boxplot

## The warning is because it automatically uses a log scale and there are some 0 count genes.
## Perhaps you prefer density plots
raw_metrics$density

## quantile/quantile plots compared to the median of all samples
raw_metrics$qqrat

raw_metrics$tsne_plot

## Here we can see some samples are differently 'shaped' compared to the median than others
## There are other plots one may view, but this data set is a bit too crowded as is.
## The following summary shows the other available plots:
summary(raw_metrics)
##                 Length Class        Mode   
## boxplot          9     gg           list   
## corheat          3     recordedplot list   
## cvplot           9     gg           list   
## density         10     gg           list   
## density_table    5     data.table   list   
## disheat          3     recordedplot list   
## gene_heatmap     0     -none-       NULL   
## legend           3     recordedplot list   
## legend_colors    3     data.frame   list   
## libsize          9     gg           list   
## libsizes         4     data.table   list   
## libsize_summary  7     data.table   list   
## ma               0     -none-       NULL   
## nonzero          9     gg           list   
## nonzero_table    7     data.frame   list   
## pc_loadplot      3     recordedplot list   
## pc_summary       4     data.frame   list   
## pc_propvar       7     -none-       numeric
## pc_plot          9     gg           list   
## pc_table        15     data.frame   list   
## qqlog            3     recordedplot list   
## qqrat            3     recordedplot list   
## smc              9     gg           list   
## smd              9     gg           list   
## topnplot         9     gg           list   
## tsne_summary     4     data.frame   list   
## tsne_propvar    20     -none-       numeric
## tsne_plot        9     gg           list   
## tsne_table      10     data.frame   list

The plots are all generated by calling plot_something() where the somethings are:

  • nonzero: scatter plot of number of non-zero genes with respect to CPM pseudocounts.
  • libsize: bar plot of the (pseudo) counts in annotated features observed by sample.
  • boxplot: boxplot describing the distribution of counts with respect to features by sample.
  • corheat: correlation heat map describing the relative similarities among samples.
  • smc: ‘standard median correlation’, take the pairwise correlations of all samples, calculate the medians, and look for a single sample with significantly lower correlations (falling below the red line).
  • disheat: distance heat map. As above but with a distance metric.
  • smd: ‘standard median distance’, as above but using the distance metrics and a line above.
  • pcaplot: plot_pca() gives many outputs, including a PCA plot of the first 2 principal components.
  • pcatable: and a table describing the same metadata.
  • pcares: along with the table of variance, cumulative variance, condition/batch r^2.
  • pcavar: and the variance by component observed.
  • density: pretty much the exact same thing as the boxplot above, but as a density plot.
  • legend: a convenient legend for figures and such.
  • qqlog: qq-plots of each sample vs. the median on the log scale.
  • qqrat: qq-plots of each sample vs. the median as ratios.
  • ma: bland-altman plots of each sample of M(log abundance) with respect to A(mean average).

6 Subsetting data

On the other hand, we might take a subset of the data to focus on the late-log vs. early-log samples.

The expt_subset() function allows one to pull material from the experimental design.

Once we have a smaller data set, we can more easily use PCA to see how the sample separate.

head(expt$design)
##          sampleid type stage replicate mutantname media exptdate  libdate batch
## HPGL0418 HPGL0418   WT    LL         1     5448-1  CDMg 20130430 20140409     a
## HPGL0419 HPGL0419   WT    LL         2     5448-2  CDMg 20130430 20140409     b
## HPGL0420 HPGL0420  mga    LL         1      Mga-1  CDMg 20130430 20140409     a
## HPGL0421 HPGL0421  mga    LL         2      Mga-2  CDMg 20130430 20140409     b
## HPGL0422 HPGL0422   WT    LL         1     5448-1  CDMf 20130430 20140415     a
## HPGL0423 HPGL0423   WT    LL         2     5448-2  CDMf 20130430 20140415     b
##           condition readspassed  ncrna xncrna remaining   genome xgenome
## HPGL0418   wt_ll_cg    17425675 350440  2.01%  17075235 16061860  94.07%
## HPGL0419   wt_ll_cg    18106361 398984  2.20%  17707377 16619183  93.85%
## HPGL0420 mga1_ll_cg    20499936 417440  2.04%  20082496 18062942  89.94%
## HPGL0421 mga1_ll_cg    17216381 397021  2.31%  16819360 14795703  87.97%
## HPGL0422   wt_ll_cf    17112706 367791  2.15%  16744915 15114930  90.27%
## HPGL0423   wt_ll_cf    18782729 395748  2.11%  18386981 17330239  94.25%
##          otherbacterial xother  colors
## HPGL0418         356542  2.09% #E12C8C
## HPGL0419         392803  2.22% #E12C8C
## HPGL0420         416260  2.07% #BE5067
## HPGL0421         376295  2.24% #BE5067
## HPGL0422         367974  2.20% #8E7E40
## HPGL0423         386443  2.10% #8E7E40
##                                                                                  counts
## HPGL0418 processed_data/count_tables/HPGL0418/HPGL0418_forward-trimmed-v1M1l20.count.xz
## HPGL0419 processed_data/count_tables/HPGL0419/HPGL0419_forward-trimmed-v1M1l20.count.xz
## HPGL0420 processed_data/count_tables/HPGL0420/HPGL0420_forward-trimmed-v1M1l20.count.xz
## HPGL0421 processed_data/count_tables/HPGL0421/HPGL0421_forward-trimmed-v1M1l20.count.xz
## HPGL0422 processed_data/count_tables/HPGL0422/HPGL0422_forward-trimmed-v1M1l20.count.xz
## HPGL0423 processed_data/count_tables/HPGL0423/HPGL0423_forward-trimmed-v1M1l20.count.xz
##          intercounts design file
## HPGL0418     unknown      a null
## HPGL0419     unknown      b null
## HPGL0420     unknown      a null
## HPGL0421     unknown      b null
## HPGL0422     unknown      a null
## HPGL0423     unknown      b null
## elt stands for: "early/late in thy"
batch_a <- subset_expt(expt, subset = "batch=='a'")
## There were 8, now there are 4 samples.
batch_b <- subset_expt(expt, subset = "batch=='b'")
## There were 8, now there are 4 samples.
a_metrics <- sm(graph_metrics(batch_a, cis = NULL))
b_metrics <- sm(graph_metrics(batch_b, cis = NULL))

PCA plots of these subsetted data are not likely to be very interesting, as it has been reduced to a single sample per condition, but we can look at them anyhow.

a_metrics$pc_plot

b_metrics$pc_plot

a_metrics$tsne_plot

b_metrics$tsne_plot

7 Normalizing data

It is pretty obvious that the raw data is a bit jumbled according to PCA. This is not paricularly suprising since we didn’t normalize it at all. Therefore, in this block I will normalize it a few ways and follow up with some visualizations of showing how the apparent relationships change in the data.

## doing nothing to the data except log2 transforming it has a surprisingly large effect
norm_test <- normalize_expt(expt, transform = "log2")
## transform_counts: Found 923 values equal to 0, adding 1 to the matrix.
l2_metrics <- sm(graph_metrics(norm_test, cis = NULL))
## a quantile normalization alone affect some, but not all of the data
norm_test <- sm(normalize_expt(expt, norm = "quant"))
q_metrics <- sm(graph_metrics(norm_test, cis = NULL))  ## q for quant, who quaffed nightshade.
## cpm alone brings out some samples, too
norm_test <- sm(normalize_expt(expt, convert = "cpm"))
c_metrics <- sm(graph_metrics(norm_test, cis = NULL))  ## c for cpm, who could not see the train.
## low count filtering has some effect, too
norm_test <- sm(normalize_expt(expt, filter = "pofa"))
f_metrics <- sm(graph_metrics(norm_test, cis = NULL))  ## f for filter, who was hit with a spade.
## how about if we mix and match methods?
norm_test <- sm(normalize_expt(expt, transform = "log2", convert = "cpm",
                               norm = "quant", batch = "combat_scale", filter = TRUE,
                               batch_step = 4, low_to_zero = TRUE))
## Some metrics are not very useful on (especially quantile) normalized data
norm_graphs <- sm(graph_metrics(norm_test, cis = NULL))

Now lets see some of the resulting metrics, in this case I will just compare some pca plots, as they are good at fooling our silly visual brains into seeing patterns.

l2_metrics$pc_plot

## Also viewable with plot_pca()$plot
## PCA plots seem (to me) to prefer log2 scale data.
q_metrics$pc_plot

## only normalizing on the quantiles leaves the data open to scale effects.
c_metrics$pc_plot

## but cpm alone is insufficient
f_metrics$pc_plot

## only filtering out low-count genes is helpful as well
norm_graphs$pc_plot

## The different batch effect testing methods have a pretty widely ranging effect on the clustering
## play with them by changing the batch= parameter to:
## "limma", "sva", "svaseq", "limmaresid", "ruvg", "combat", combatmod"
knitr::kable(norm_graphs$pc_summary)
prop_var cum_prop_var condition_rsquared batch_rsquared
79.29 79.29 0.9970 0.0000
9.81 89.10 0.9849 0.0023
4.58 93.68 0.3403 0.1100
2.99 96.67 0.6224 0.0094
1.77 98.44 0.0351 0.0019
0.84 99.28 0.0092 0.5731
0.73 100.01 0.0111 0.3033
## Thus we see a dramatic decrease in variance accounted for
## by batch after applying limma's 'removebatcheffect'
## (see batch.R2 here vs. above)
norm_graphs$smc

norm_graphs$disheat  ## svaseq's batch correction seems to draw out the signal quite nicely.

## It is worth noting that the wt, early log, thy, replicate c samples are still a bit weird.
norm_graphs$tsne_plot

8 Performing DE analyses

This is a relatively small data set, so performing some differential expression analyses really should not take long at all.

When performing these analyses with hpgltools, it will attempt to perform similar analyses with limma, edgeR, and DESeq2 via the all_pairwise() function. The most likely argument is ‘model_batch’ which may be used to explicitly include/exclude a batch factor in the model, or ask it to attempt including batch factors from sva/ruv/etc. By default it will attempt to include a column from the experimental design named ‘batch’.

spyogenes_de <- sm(all_pairwise(expt))

## Even the lowest correlations are quite high.

The result of all_pairwise() is a list of the results from limma, edger, and deseq. In addition, I implemented a very simplistic, differential expression function named ‘basic()’. It also provides some simple measurements of how well the various analyses agree (ergo the black and white heatmap).

Working with these separate tables can be more than a little annoying, combine_de_tables() attempts to simplify this. It will bring together the various tables, and if asked attempt to bring them together into a pretty-ified excel workbook.

all_pairwise() arbitrarily performs all possible pairwise comparisons. This is not necessarily what one actually wishes to see. Therefore, the argument keepers takes a list of contrasts:

my_keepers <- list(
  ## name    =   numerator / denominator
  "wt_media" = c("wt_ll_cf", "wt_ll_cg"),
  "mga_media" = c("mga_ll_cf", "mga_ll_cg"))

In the above example, if the keepers argument to combine_de_tables() is given as my_keepers, then the resulting table will not have the set of 6 possible comparisons, but instead will only have 2 tables named ‘wt_media’ and ‘mga_media’, which if printed to excel will be sheets named accordingly.

spyogenes_tables <- sm(combine_de_tables(spyogenes_de, excel = FALSE))
summary(spyogenes_tables)
##             Length Class        Mode     
## data         6     -none-       list     
## table_names  6     -none-       list     
## input       14     all_pairwise list     
## plots        6     -none-       list     
## comp_plot    0     -none-       list     
## venns        0     -none-       list     
## keepers      1     -none-       character
## kept         6     -none-       list     
## de_summary  22     data.frame   list
## Try changing the p-adjustment
spyogenes_tables <- sm(combine_de_tables(spyogenes_de, excel = FALSE, padj_type = "BH"))
## Error in `[.data.frame`(combined_data, , columns): undefined columns selected
knitr::kable(head(spyogenes_tables$data[[1]]))
seqnames start end width strand source type score phase id dbxref iscircular gbkey genome moltype strain name note gene locustag parent product proteinid transltable genesynonym deseq_logfc deseq_adjp edger_logfc edger_adjp limma_logfc limma_adjp basic_nummed basic_denmed basic_numvar basic_denvar basic_logfc basic_t basic_p basic_adjp deseq_basemean deseq_lfcse deseq_stat deseq_p ebseq_fc ebseq_logfc ebseq_c1mean ebseq_c2mean ebseq_mean ebseq_var ebseq_postfc ebseq_ppee ebseq_ppde ebseq_adjp edger_logcpm edger_lr edger_p limma_ave limma_t limma_b limma_p limma_adjp_ihw deseq_adjp_ihw edger_adjp_ihw ebseq_adjp_ihw basic_adjp_ihw lfc_meta lfc_var lfc_varbymed p_meta p_var
M5005_Spy0001 NC_007297 202 1557 1356 + RefSeq gene undefined undefined M5005_Spy0001 GeneID:3571011 undefined Gene undefined undefined undefined dnaA M5005_Spy0001 dnaA M5005_Spy_0001 character(0) undefined undefined undefined character(0) 0.2219 0.6393 0.2369 0.6469 0.2232 0.5637 9.801 9.581 0.0739 0.0095 0.2193 1.0740 0.4471 0.7454 8667.0 0.2008 1.1050 0.2690 1.1480 0.1992 8765.6 10063.3 9414.5 1635329 1.1480 0.9742 0.0258 0.9742 9.562 1.4830 0.2234 9.525 1.3760 -6.173 0.2286 5.636e-01 7.297e-01 6.468e-01 4.403e-02 6.873e-01 0.2272 7.468e-06 3.287e-05 2.403e-01 6.231e-04
M5005_Spy0002 NC_007297 1712 2848 1137 + RefSeq gene undefined undefined M5005_Spy0002 GeneID:3571012 undefined Gene undefined undefined undefined dnaN M5005_Spy0002 dnaN M5005_Spy_0002 character(0) undefined undefined undefined character(0) 0.1595 0.7990 0.1747 0.7889 0.1799 0.7002 9.774 9.596 0.0258 0.0115 0.1781 1.3030 0.3383 0.6858 9174.0 0.2240 0.7121 0.4764 1.0905 0.1250 8863.1 9664.9 9264.0 523572 1.0905 0.9636 0.0364 0.9636 9.643 0.7114 0.3990 9.621 0.9257 -6.657 0.3981 7.002e-01 8.856e-01 7.890e-01 5.696e-02 6.072e-01 0.1723 2.446e-05 1.420e-04 4.245e-01 2.020e-03
M5005_Spy0003 NC_007297 2923 3120 198 + RefSeq gene undefined undefined M5005_Spy0003 GeneID:3571013 undefined Gene undefined undefined undefined M5005_Spy_0003 M5005_Spy0003 undefined M5005_Spy_0003 character(0) undefined undefined undefined character(0) -0.0451 0.9706 -0.0307 1.0000 -0.0661 0.9421 6.242 6.311 0.0599 0.0002 -0.0693 -0.4000 0.7575 0.9030 699.7 0.2304 -0.1957 0.8448 0.9466 -0.0791 874.5 827.8 851.2 6016 0.9467 0.9770 0.0230 0.9770 5.935 0.0141 0.9054 5.855 -0.3763 -6.771 0.7225 9.421e-01 1.000e+00 1.000e+00 4.085e-02 8.815e-01 -0.0533 4.066e-04 -7.630e-03 8.242e-01 8.680e-03
M5005_Spy0004 NC_007297 3450 4565 1116 + RefSeq gene undefined undefined M5005_Spy0004 GeneID:3571014 undefined Gene undefined undefined undefined ychF M5005_Spy0004 ychF M5005_Spy_0004 character(0) undefined undefined undefined character(0) 0.0541 0.9578 0.0685 0.9715 0.0642 0.9447 10.510 10.440 0.0572 0.0271 0.0628 0.3058 0.7918 0.9192 12590.0 0.2148 0.2516 0.8013 1.0109 0.0157 15562.2 15732.0 15647.1 2602409 1.0109 0.9933 0.0067 0.9933 10.100 0.1190 0.7302 10.060 0.3657 -7.081 0.7299 9.447e-01 1.000e+00 9.715e-01 1.670e-02 9.008e-01 0.0622 2.212e-06 3.557e-05 7.538e-01 1.692e-03
M5005_Spy0005 NC_007297 4635 5204 570 + RefSeq gene undefined undefined M5005_Spy0005 GeneID:3571015 undefined Gene undefined undefined undefined pth M5005_Spy0005 pth M5005_Spy_0005 character(0) undefined undefined undefined character(0) 0.0526 0.9646 0.0676 0.9980 0.0745 0.9169 6.651 6.576 0.0113 0.0048 0.0750 0.8344 0.5038 0.7620 1017.0 0.2201 0.2391 0.8111 1.0107 0.0154 1069.0 1080.5 1074.7 3041 1.0107 0.9914 0.0086 0.9914 6.473 0.0703 0.7909 6.452 0.4288 -6.768 0.6863 9.169e-01 1.000e+00 9.979e-01 2.049e-02 7.156e-01 0.0638 1.499e-05 2.351e-04 7.628e-01 4.487e-03
M5005_Spy0006 NC_007297 5207 8710 3504 + RefSeq gene undefined undefined M5005_Spy0006 GeneID:3571016 undefined Gene undefined undefined undefined trcF M5005_Spy0006 trcF M5005_Spy_0006 character(0) undefined undefined undefined character(0) -0.0932 0.8750 -0.0786 0.9398 -0.1120 0.6719 10.020 10.140 0.0399 0.0028 -0.1157 -0.7922 0.5600 0.7995 11000.0 0.1784 -0.5228 0.6011 0.9148 -0.1284 12550.4 11481.3 12015.9 790407 0.9148 0.9700 0.0300 0.9700 9.904 0.1844 0.6676 9.907 -0.9875 -6.613 0.3698 6.719e-01 9.536e-01 9.397e-01 4.909e-02 7.584e-01 -0.0958 6.661e-04 -6.956e-03 5.462e-01 2.443e-02

Finally, extract_significant_genes() may choose ‘significant’ genes based upon a few metrics including z-score vs. the distribution of logFC; a logFC cutoff, (ajusted)p-value cutoff, and/or top/bottom n genes.

spyogenes_sig <- sm(extract_significant_genes(spyogenes_tables, excel = FALSE))
knitr::kable(head(spyogenes_sig$limma$ups[[1]]))
seqnames start end width strand source type score phase id dbxref iscircular gbkey genome moltype strain name note gene locustag parent product proteinid transltable genesynonym deseq_logfc deseq_adjp edger_logfc edger_adjp limma_logfc limma_adjp basic_nummed basic_denmed basic_numvar basic_denvar basic_logfc basic_t basic_p basic_adjp deseq_basemean deseq_lfcse deseq_stat deseq_p ebseq_fc ebseq_logfc ebseq_c1mean ebseq_c2mean ebseq_mean ebseq_var ebseq_postfc ebseq_ppee ebseq_ppde ebseq_adjp edger_logcpm edger_lr edger_p limma_ave limma_t limma_b limma_p limma_adjp_ihw deseq_adjp_ihw edger_adjp_ihw ebseq_adjp_ihw basic_adjp_ihw lfc_meta lfc_var lfc_varbymed p_meta p_var
M5005_Spy1735 NC_007297 1696949 1698145 1197 - RefSeq gene undefined undefined M5005_Spy1735 GeneID:3571136 undefined Gene undefined undefined undefined speB M5005_Spy1735 speB M5005_Spy_1735 character(0) undefined undefined undefined character(0) 3.691 0 3.696 0 3.509 0.0356 4.349 1.067 0.0350 0.0215 3.282 19.51 0.0034 0.6231 83.91 0.4340 8.504 0 12.600 3.655 15.80 199.2 107.52 11221 11.998 0 1 0 2.925 101.70 0 2.346 13.17 1.865 1e-04 3.555e-02 1.172e-14 6.258e-21 1.000e+00 2.583e-02 3.710 1.708e-01 4.602e-02 1.778e-05 9.487e-10
M5005_Spy1575 NC_007297 1535635 1536831 1197 - RefSeq gene undefined undefined M5005_Spy1575 GeneID:3571303 undefined Gene undefined undefined undefined norA M5005_Spy1575 norA M5005_Spy_1575 character(0) undefined undefined undefined character(0) 2.406 0 2.422 0 2.371 0.0356 8.250 5.881 0.0368 0.0561 2.368 10.99 0.0095 0.6231 1849.00 0.2276 10.570 0 5.187 2.375 632.69 3281.9 1957.31 2466670 5.181 0 1 0 7.338 87.72 0 6.834 12.96 2.727 1e-04 3.555e-02 3.870e-23 4.838e-18 1.000e+00 6.863e-02 2.507 8.927e-02 3.561e-02 1.921e-05 1.107e-09
M5005_Spy1715 NC_007297 1675257 1678751 3495 - RefSeq gene undefined undefined M5005_Spy1715 GeneID:3571155 undefined Gene undefined undefined undefined scpA M5005_Spy1715 scpA M5005_Spy_1715 character(0) undefined undefined undefined character(0) 1.837 0 1.850 0 1.780 0.0359 3.909 2.155 0.0033 0.0018 1.754 35.03 0.0013 0.6231 1182.00 0.2667 6.888 0 3.489 1.803 43.05 150.2 96.64 3892 3.440 0 1 0 6.681 39.32 0 5.291 11.71 1.906 1e-04 3.594e-02 1.556e-09 7.695e-08 1.000e+00 1.033e-02 1.820 1.114e-01 6.122e-02 3.110e-05 2.901e-09

9 Make pretty circos graphs

Since most of my circos graphs are for pyogenes, it is likely that the defaults are appropriate for this particular organism.

Much(all) of the following is taken from the material in tests/testthat/test_70mga.R

##microbe_ids <- as.character(sm(get_microbesonline_ids("pyogenes MGAS5005")))
## A caveat!  The new version of microbesonline changed the IDs so that they no longer
## match my old rnaseq analysis!!  Thus I put my old gff file used for mapping into inst/
## and will load the annotation data from that; but I will use this data to gather
## the COG information.
mgas_gff_df <- sm(load_gff_annotations(gff = system.file("share/gas.gff", package = "hpgltools")))
mgas_gff_df <- mgas_gff_df[-1, ]
mgas_gff_df[["sysName"]] <- gsub(pattern = "Spy_", replacement = "Spy",
                                 x = mgas_gff_df[["locus_tag"]])
rownames(mgas_gff_df) <- make.names(mgas_gff_df[["sysName"]], unique = TRUE)

mgas_microbes_df <- sm(load_microbesonline_annotations(id = 293653))
mgas_microbes_df$sysName <- gsub(pattern = "Spy_", replacement = "Spy",
                                 x = mgas_microbes_df$sysName)
rownames(mgas_microbes_df) <- make.names(mgas_microbes_df$sysName, unique = TRUE)
## Warning: Setting row names on a tibble is deprecated.
mgas_df <- merge(x = mgas_gff_df, y = mgas_microbes_df, by = "row.names")
rownames(mgas_df) <- mgas_df[["Row.names"]]
mgas_df <- mgas_df[, -1]
colnames(mgas_df) <- c("seqnames", "start", "end", "width", "strand", "source", "type",
                       "score", "phase", "ID", "Dbxref", "Is_circular", "gbkey", "genome",
                       "mol_type", "strain", "Name", "Note", "gene", "locus_tag",
                       "Parent", "product", "protein_id", "transl_table", "gene_synonym",
                       "sysName_again", "locusId", "accession", "GI", "scaffoldId",
                       "start_again", "stop", "strand_again", "sysName_again", "name",
                       "desc", "COG", "COGFun", "COGDesc", "TIGRFam", "TIGRRoles",
                       "GO", "EC", "ECDesc")

## First make a template configuration
circos_test <- circos_prefix(annotation = mgas_df)
## This assumes you have a colors.conf in circos/colors/ and fonts.conf in circos/fonts/
## It also assumes you have conf/ideogram.conf, conf/ticks.conf, and conf/housekeeping.conf
## It will write circos/conf/mgas.conf with a reasonable first approximation config file.
## Creating the data directory: circos/data
## The circos directory does not exist, creating: circos/conf
## The karyotype directory does not exist, creating: circos/conf/karyotypes
## The ideogram directory does not exist, creating: circos/conf/ideograms
## Wrote karyotype to circos/conf/ideograms/mgas.conf
## This should match the karyotype= line in mgas.conf
## Wrote ticks to circos/conf/ticks_mgas.conf
## Fill it in with the data for s.pyogenes
lengths <- 1838600
names(lengths) <- "NC_007297"

circos_kary <- circos_karyotype(cfg = circos_test, lengths = lengths)
## Wrote karyotype to circos/conf/karyotypes/mgas.conf
## This should match the karyotype= line in mgas.conf
## Fill in the gene category annotations by gene-strand
circos_plus <- circos_plus_minus(cfg = circos_test)
## Writing data file: circos/data/mgas_plus_go.txt with the + strand GO data.
## Writing data file: circos/data/mgas_minus_go.txt with the - strand GO data.
## Wrote the +/- config files.  Appending their inclusion to the master file.
## Returning the inner width: 0.84.  Use it as the outer for the next ring.
circos_limma_hist <- circos_hist(cfg = circos_test,
                                 df = spyogenes_de$limma$all_tables[[1]],
                                 basename = "limma",
                                 colname = "logFC",
                                 outer = circos_plus)
## Writing data file: circos/data/mgas_limmalogFC_hist.txt with the limmalogFC column.
## Returning the inner width: 0.76.  Use it as the outer for the next ring.
circos_deseq_hist <- circos_hist(cfg = circos_test,
                                 df = spyogenes_de$deseq$all_tables[[1]],
                                 basename = "deseq",
                                 colname = "logFC",
                                 outer = circos_limma_hist)
## Writing data file: circos/data/mgas_deseqlogFC_hist.txt with the deseqlogFC column.
## Returning the inner width: 0.68.  Use it as the outer for the next ring.
circos_edger_hist <- circos_hist(cfg = circos_test,
                                 df = spyogenes_de$edger$all_tables[[1]],
                                 basename = "edger",
                                 colname = "logFC",
                                 outer = circos_deseq_hist)
## Writing data file: circos/data/mgas_edgerlogFC_hist.txt with the edgerlogFC column.
## Returning the inner width: 0.6.  Use it as the outer for the next ring.
circos_suffix(cfg = circos_test)
circos_made <- sm(circos_make(cfg = circos_test, target = "mgas"))
getwd()
## [1] "/mnt/sshfs_10186/cbcbsub01/fs/cbcb-lab/nelsayed/scratch/atb/git/hpgltools/vignettes"

circos result

GenoplotR is new to me, but it seems to work?

genoplot_chromosome()

10 Change conditions

Perhaps instead of looking at subsets of the data, we may want to consider wt/mga. If so, we can just change the condition to any column in the design matrix.

wt_mga_expt <- set_expt_conditions(expt = expt, fact = "type")
wt_mga_plots <- sm(graph_metrics(wt_mga_expt))
wt_mga_norm <- sm(normalize_expt(wt_mga_expt, transform = "log2", convert = "raw", filter = TRUE, norm = "quant"))
wt_mga_nplots <- sm(graph_metrics(wt_mga_norm))
wt_mga_de <- sm(all_pairwise(input = wt_mga_expt,
                          combined_excel = "wt_mga.xlsx",
                          sig_excel = "wt_mga_sig.xlsx",
                          abundant_excel = "wt_mga_abundant.xlsx"))
wt_mga_de$combined$comp_plot
## list()
## How well do the various DE tools agree on this data?

wt_mga_plots$tsne_plot

wt_mga_nplots$pc_plot

wt_mga_de$combined$limma_plots$WT_vs_mga$scatter
## NULL
wt_mga_de$combined$limma_ma_plots$WT_vs_mga$plot
## NULL
wt_mga_de$combined$limma_vol_plots$WT_vs_mga$plot
## NULL
pander::pander(sessionInfo())

R version 4.0.3 (2020-10-10)

Platform: x86_64-pc-linux-gnu (64-bit)

locale: LC_CTYPE=en_US.UTF-8, LC_NUMERIC=C, LC_TIME=en_US.UTF-8, LC_COLLATE=en_US.UTF-8, LC_MONETARY=en_US.UTF-8, LC_MESSAGES=en_US.UTF-8, LC_PAPER=en_US.UTF-8, LC_NAME=C, LC_ADDRESS=C, LC_TELEPHONE=C, LC_MEASUREMENT=en_US.UTF-8 and LC_IDENTIFICATION=C

attached base packages: stats4, parallel, stats, graphics, grDevices, utils, datasets, methods and base

other attached packages: ruv(v.0.9.7.1), SummarizedExperiment(v.1.20.0), GenomicRanges(v.1.42.0), GenomeInfoDb(v.1.26.2), IRanges(v.2.24.1), S4Vectors(v.0.28.1), MatrixGenerics(v.1.2.1), matrixStats(v.0.58.0), hpgltools(v.1.0), R6(v.2.5.0), Biobase(v.2.50.0) and BiocGenerics(v.0.36.0)

loaded via a namespace (and not attached): Rtsne(v.0.15), colorspace(v.2.0-0), selectr(v.0.4-2), ellipsis(v.0.3.1), corpcor(v.1.6.9), XVector(v.0.30.0), Vennerable(v.3.1.0.9000), rstudioapi(v.0.13), farver(v.2.0.3), ggrepel(v.0.9.1), bit64(v.4.0.5), AnnotationDbi(v.1.52.0), fansi(v.0.4.2), xml2(v.1.3.2), R.methodsS3(v.1.8.1), codetools(v.0.2-18), splines(v.4.0.3), doParallel(v.1.0.16), cachem(v.1.0.4), robustbase(v.0.93-7), geneplotter(v.1.68.0), knitr(v.1.31), ade4(v.1.7-16), jsonlite(v.1.7.2), Rsamtools(v.2.6.0), annotate(v.1.68.0), R.oo(v.1.24.0), graph(v.1.68.0), readr(v.1.4.0), compiler(v.4.0.3), httr(v.1.4.2), assertthat(v.0.2.1), Matrix(v.1.3-2), fastmap(v.1.1.0), cli(v.2.3.1), limma(v.3.46.0), prettyunits(v.1.1.1), htmltools(v.0.5.1.1), tools(v.4.0.3), gtable(v.0.3.0), glue(v.1.4.2), GenomeInfoDbData(v.1.2.4), reshape2(v.1.4.4), dplyr(v.1.0.4), Rcpp(v.1.0.6), slam(v.0.1-48), jquerylib(v.0.1.3), vctrs(v.0.3.6), Biostrings(v.2.58.0), preprocessCore(v.1.52.1), nlme(v.3.1-152), rtracklayer(v.1.50.0), iterators(v.1.0.13), xfun(v.0.21), fastcluster(v.1.1.25), stringr(v.1.4.0), ps(v.1.5.0), rvest(v.0.3.6), openxlsx(v.4.2.3), lifecycle(v.1.0.0), gtools(v.3.8.2), XML(v.3.99-0.5), edgeR(v.3.32.1), DEoptimR(v.1.0-8), MASS(v.7.3-53.1), directlabels(v.2021.1.13), zlibbioc(v.1.36.0), scales(v.1.1.1), hms(v.1.0.0), RBGL(v.1.66.0), RColorBrewer(v.1.1-2), curl(v.4.3), yaml(v.2.2.1), memoise(v.2.0.0), gridExtra(v.2.3), pander(v.0.6.3), ggplot2(v.3.3.3), sass(v.0.3.1), stringi(v.1.5.3), RSQLite(v.2.2.3), highr(v.0.8), genefilter(v.1.72.1), foreach(v.1.5.1), caTools(v.1.18.1), zip(v.2.1.1), BiocParallel(v.1.24.1), rlang(v.0.4.10), pkgconfig(v.2.0.3), bitops(v.1.0-6), evaluate(v.0.14), lattice(v.0.20-41), lpsymphony(v.1.18.0), purrr(v.0.3.4), GenomicAlignments(v.1.26.0), labeling(v.0.4.2), bit(v.4.0.4), tidyselect(v.1.1.0), plyr(v.1.8.6), magrittr(v.2.0.1), DESeq2(v.1.30.1), gplots(v.3.1.1), IHW(v.1.18.0), generics(v.0.1.0), DelayedArray(v.0.16.1), DBI(v.1.1.1), pillar(v.1.5.0), mgcv(v.1.8-34), survival(v.3.2-7), RCurl(v.1.98-1.2), tibble(v.3.0.6), crayon(v.1.4.1), fdrtool(v.1.2.16), KernSmooth(v.2.23-18), utf8(v.1.1.4), rmarkdown(v.2.7), progress(v.1.2.2), locfit(v.1.5-9.4), grid(v.4.0.3), sva(v.3.38.0), data.table(v.1.14.0), blob(v.1.2.1), digest(v.0.6.27), xtable(v.1.8-4), genoPlotR(v.0.8.11), R.utils(v.2.10.1), munsell(v.0.5.0), bslib(v.0.2.4) and quadprog(v.1.5-8)

LS0tCnRpdGxlOiAiSHBnbHRvb2xzIGV4YW1wbGVzIHVzaW5nIGEgc21hbGwgYmFjdGVyaWFsIGRhdGEgc2V0LiIKYXV0aG9yOiAiYXRiIGFiZWxld0BnbWFpbC5jb20iCmRhdGU6ICJgciBTeXMuRGF0ZSgpYCIKb3V0cHV0OgogaHRtbF9kb2N1bWVudDoKICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgY29kZV9mb2xkaW5nOiBzaG93CiAgZmlnX2NhcHRpb246IHRydWUKICBmaWdfaGVpZ2h0OiA3CiAgZmlnX3dpZHRoOiA3CiAgaGlnaGxpZ2h0OiBkZWZhdWx0CiAga2VlcF9tZDogZmFsc2UKICBtb2RlOiBzZWxmY29udGFpbmVkCiAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCiAgc2VsZl9jb250YWluZWQ6IHRydWUKICB0aGVtZTogcmVhZGFibGUKICB0b2M6IHRydWUKICB0b2NfZmxvYXQ6CiAgICBjb2xsYXBzZWQ6IGZhbHNlCiAgICBzbW9vdGhfc2Nyb2xsOiBmYWxzZQp2aWduZXR0ZTogPgogICVcVmlnbmV0dGVJbmRleEVudHJ5e2EtMDFfYmFjdGVyaWFsX2V4YW1wbGV9CiAgJVxWaWduZXR0ZUVuZ2luZXtrbml0cjo6cm1hcmtkb3dufQogIFx1c2VwYWNrYWdlW3V0Zjhde2lucHV0ZW5jfQotLS0KCjxzdHlsZT4KICBib2R5IC5tYWluLWNvbnRhaW5lciB7CiAgICBtYXgtd2lkdGg6IDE2MDBweDsKICB9Cjwvc3R5bGU+CgpgYGB7ciBvcHRpb25zLCBpbmNsdWRlID0gRkFMU0V9CmxpYnJhcnkoImhwZ2x0b29scyIpCmtuaXRyOjpvcHRzX2tuaXQkc2V0KHByb2dyZXNzID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZSA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICAgIHdpZHRoID0gOTAsCiAgICAgICAgICAgICAgICAgICAgIGVjaG8gPSBUUlVFKQprbml0cjo6b3B0c19jaHVuayRzZXQoZXJyb3IgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICAgZmlnLndpZHRoID0gOCwKICAgICAgICAgICAgICAgICAgICAgIGZpZy5oZWlnaHQgPSA4LAogICAgICAgICAgICAgICAgICAgICAgZHBpID0gOTYpCm9sZF9vcHRpb25zIDwtIG9wdGlvbnMoZGlnaXRzID0gNCwKICAgICAgICAgICAgICAgICAgICAgICBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICAga25pdHIuZHVwbGljYXRlLmxhYmVsID0gImFsbG93IikKZ2dwbG90Mjo6dGhlbWVfc2V0KGdncGxvdDI6OnRoZW1lX2J3KGJhc2Vfc2l6ZSA9IDEwKSkKc2V0LnNlZWQoMSkKcm1kX2ZpbGUgPC0gImEtMDFfYmFjdGVyaWFsX2V4YW1wbGUuUm1kIgpgYGAKCiMgTG9hZGluZyAobWV0YSlkYXRhIGFuZCBhbm5vdGF0aW9ucwoKVGhlIGZvbGxvd2luZyBleGFtcGxlcyB3aWxsIHVzZSBhIHJlYWwgZGF0YSBzZXQgZnJvbSBhbgpleHBlcmltZW50IGluIG91ciBsYWIuICBUaGUgcmF3IGRhdGEgd2FzIHByb2Nlc3NlZCB1c2luZyBhIG1peCBvZgp0cmltbW9tYXRpYywgYmlvcGllY2VzLCBib3d0aWUsIHNhbXRvb2xzLCBhbmQgaHRzZXEuICBUaGUgZmluYWwgY291bnQKdGFibGVzIHdlcmUgZGVwb3NpdGVkIGludG8gdGhlICdwcmVwcm9jZXNzaW5nL2NvdW50X3RhYmxlcy8nIHRyZWUuClRoZSByZXN1bHRpbmcgZGF0YSBzdHJ1Y3R1cmUgd2FzIG5hbWVkICdtb3N0X3YwTTEsJyBuYW1lZCBiZWNhdXNlIGl0CmlzIGNvbXByaXNlZCBvZiBjb3VudCB0YWJsZXMgd2l0aCAwIG1pc21hdGNoZXMgYW5kIDEgcmFuZG9tbHktcGxhY2VkCm11bHRpLW1hdGNoLgoKVGhlIGFubm90YXRpb24gZmlsZSB3YXMgbWdhc181MDA1LmdmZi54eiByZXNpZGVzIGluCidyZWZlcmVuY2UvZ2ZmLycuCgpUaGUgY291bnQgdGFibGVzIGFuZCBtZXRhLWRhdGEgd2VyZSBsb2FkZWQgdGhyb3VnaCB0aGUgY3JlYXRlX2V4cHQoKQpmdW5jdGlvbiBhbmQgdGhlIGdlbm9tZSBhbm5vdGF0aW9ucyB3ZXJlIGxvYWRlZCB3aXRoIGdmZjJkZigpLgoKYGBge3IgbG9hZGluZ19kYXRhfQpsaWJyYXJ5KGhwZ2x0b29scykKZGF0YV9maWxlIDwtIHN5c3RlbS5maWxlKCJzaGFyZS9jZG1fZXhwdC5yZGEiLCBwYWNrYWdlID0gImhwZ2x0b29scyIpCmNkbSA8LSBuZXcuZW52KCkKbG9hZChkYXRhX2ZpbGUsIGVudmlyID0gY2RtKQpybShkYXRhX2ZpbGUpCgpscygpCmBgYAoKVHdvIHZhcmlhYmxlcyBzaG91bGQgZXhpc3Qgbm93OiBybWRfZmlsZSBpbiBjYXNlIEkgd2FudCB0byBrbml0ciB0aGlzIGZpbGUsIGNkbSB3aGljaCBpcyBhIGxpc3QKaW5jbHVkaW5nIHRoZSBkYXRhIHJlcXVpcmVkIHRvIG1ha2UgYW4gZXhwcmVzc2lvbnNldC4gIFVzaW5nIHRoaXMgaW5mb3JtYXRpb24sIEkgY2FuIGNyZWF0ZSBhbgpleHByZXNzaW9uc2V0LgoKYGBge3IgY3JlYXRlX2V4cHR9CmV4cHQgPC0gY3JlYXRlX2V4cHQoY291bnRfZGF0YWZyYW1lID0gY2RtJGNkbV9jb3VudHMsCiAgICAgICAgICAgICAgICAgICAgbWV0YWRhdGEgPSBjZG0kY2RtX21ldGFkYXRhLAogICAgICAgICAgICAgICAgICAgIGdlbmVfaW5mbyA9IGNkbSRnZW5lX2luZm8pCgprbml0cjo6a2FibGUoaGVhZChleHB0JGRlc2lnbikpCnN1bW1hcnkoZXhwdCkKYGBgCgojIEludHJvZHVjdGlvbgoKaHBnbHRvb2xzIHdhcyB3cml0dGVuIHRvIG1ha2Ugd29ya2luZyB3aXRoIGhpZ2gtdGhyb3VnaHB1dCBkYXRhCmFuYWx5c2VzIGVhc2llci4gVGhlc2UgYW5hbHlzZXMgZ2VuZXJhbGx5IGZhbGwgaW50byBhIGZldyBzdGFnZXM6CgoxLiAgRGF0YSB2aXN1YWxpemF0aW9uIGFuZCBvdXRsaWVyL2JhdGNoIGV2YWx1YXRpb24KMi4gIERpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2VzCiAgYS4gVmlzdWFsaXphdGlvbiBhbmQgZXhwb3J0IG9mIHRoZXNlIHJlc3VsdHMKMy4gIEdlbmUgb250b2xvZ3kvS0VHRyBhbmFseXNlcwogIGEuIFZpc3VhbGl6YXRpb24gYW5kIGV4cG9ydCBvZiB0aGVzZSByZXN1bHRzCjQuICBHZW5vbWUgdmlzdWFsaXphdGlvbnMKICBhLiBXaXRoIGNpcmNvcwogIGIuIFdpdGggZ2Vub3Bsb3RSCgpCZWZvcmUgYW55IG9mIHRoZXNlIHRhc2tzIG1heSBiZSBwZXJmb3JtZWQsIHRoZSBkYXRhIG11c3QgYmUgbG9hZGVkCmludG8gbWVtb3J5LiAgaHBnbHRvb2xzIGF0dGVtcHRzIHRvIG1ha2UgdGhpcyBlYXNpZXIgd2l0aApjcmVhdGVfZXhwdCgpIGFuZCBzdWJzZXRfZXhwdCgpLgoKIyBBIGxpdHRsZSBzaWRlLXRyYWNrCgpJIHdhbnQgdG8gdXNlIHRoaXMgdmlnbmV0dGUgdG8gZXhwbG9yZSB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gZXhwcmVzc2lvblNldHMKYW5kIFN1bW1hcml6ZWRFeHBlcmltZW50cy4gIEkgdXNlIHRoZSBmb3JtZXIgZXh0ZW5zaXZlbHksIGFuZCBoYXZlIG5vdGljZWQgYnV0Cm5vdCBwYWlkIHN0cm9uZyBhdHRlbnRpb24gdG8gdGhlIGxhdHRlci4KCmBgYHtyIHRlc3Rfc2V9CnR0IDwtIHNtKGxpYnJhcnkoU3VtbWFyaXplZEV4cGVyaW1lbnQpKQp0ZXN0X2V4cHIgPC0gZXhwdFtbImV4cHJlc3Npb25zZXQiXV0KCiMjIFRoZSBmb2xsb3dpbmcgaW4gdGhlb3J5IGNvbnZlcnRzIHRvIGEgU0UsIGJ1dCBpcyBhIGJpdCB1bmhlbHBmdWwgaW4gaXRzIHJlc3VsdC4KIyMgQ29udmVydCBleHByZXNzaW9uU2V0IHRvIHN1bW1hcml6ZWRFeHBlcmltZW50CnRlc3Rfc2UgPC0gU3VtbWFyaXplZEV4cGVyaW1lbnQ6Om1ha2VTdW1tYXJpemVkRXhwZXJpbWVudEZyb21FeHByZXNzaW9uU2V0KHRlc3RfZXhwcikKIyMgSG9seSBjcmFwIGxvbmcgZnVuY3Rpb24gbmFtZSEKdGVzdF9hc3NheSA8LSBhc3NheSh0ZXN0X3NlKQpoZWFkKHRlc3RfYXNzYXkpCnRlc3RfbWV0YSA8LSBjb2xEYXRhKHRlc3Rfc2UpCmhlYWQodGVzdF9tZXRhKQp0ZXN0X2dlbmVfaW5mbyA8LSByb3dEYXRhKHRlc3Rfc2UpCmhlYWQodGVzdF9nZW5lX2luZm8pCgojIyBIZXJlIGlzIGEgc3R1cGlkZXIgdmVyc2lvbiBvZiB0aGUgYWJvdmUsIGJ1dCBvbmUgd2hpY2ggd2lsbCBub3QgbWFrZSBteSBmaW5nZXJzIHNhZC4KZXhwcnNfdG9fc2UgPC0gZnVuY3Rpb24oZXhzZXQpIHsKICBtdHJ4IDwtIEJpb2Jhc2U6OmV4cHJzKGV4c2V0KQogIGFubm90IDwtIEJpb2Jhc2U6OmZEYXRhKGV4c2V0KQogIG1ldGEgPC0gQmlvYmFzZTo6cERhdGEoZXhzZXQpCiAgc2UgPC0gU3VtbWFyaXplZEV4cGVyaW1lbnQoYXNzYXlzID0gbXRyeCwgY29sRGF0YSA9IG1ldGEsIHJvd0RhdGEgPSBhbm5vdCkKICByZXR1cm4oc2UpCn0KCiMjIGFuZCBiYWNrPwp0ZXN0X2V4IDwtIGFzKHRlc3Rfc2UsICJFeHByZXNzaW9uU2V0IikKaGVhZChleHBycyh0ZXN0X2V4KSkKCiMjIFlBWSEKYGBgCgojIFdvcmtpbmcgd2l0aCBleHB0cwoKVGhlIGRhdGEgc3RydWN0dXJlIGdlbmVyYXRlZCBieSBjcmVhdGVfZXhwdCgpIGlzIGEgbGlzdCBjb250YWluaW5nIHRoZQpmb2xsb3dpbmcgc2xvdHM6CgoqIGluaXRpYWxfbWV0YWRhdGE6ICAgICAgICBBIGJhY2t1cCBvZiB0aGUgbWV0YWRhdGEKKiBvcmlnaW5hbF9leHByZXNzaW9uc2V0OiAgQSBiYWNrdXAgb2YgdGhlIHJhdyBjb3VudHMKKiBleHByZXNzaW9uc2V0OiAgICAgICAgICAgVGhlIGN1cnJlbnQgY291bnQgZGF0YQoqIHNhbXBsZXM6ICAgICAgICAgICAgICAgICBBIGRhdGEgZnJhbWUgb2YgbWV0YWRhdGEgdXNlZCBmb3Igc3Vic2V0cwoqIGRlc2lnbjogICAgICAgICAgICAgICAgICBUaGUgZGVzaWduIG9mIHRoZSBleHBlcmltZW50CiogZGVmaW5pdGlvbnM6ICAgICAgICAgICAgIEV4dGVuZGVkIGRlc2lnbiBpbmZvcm1hdGlvbiwgdGhlc2UgYXJlCiAgcHJvYmFibHkgcmVkdW5kYW50IGFuZCBzaG91bGQgYmUgcHJ1bmVkLgoqIHN0YWdlczogICAgICAgICAgICAgICAgICBUaGUgZXhwZXJpbWVudGFsIHN0YWdlCiogdHlwZXM6ICAgICAgICAgICAgICAgICAgIENlbGwgdHlwZXMKKiBjb25kaXRpb25zOiAgICAgICAgICAgICAgRXhwZXJpbWVudGFsIGNvbmRpdGlvbgoqIGJhdGNoZXM6ICAgICAgICAgICAgICAgICBFeHBlcmltZW50YWwgYmF0Y2gKKiBzYW1wbGVuYW1lczogICAgICAgICAgICAgTmFtZXMgb2YgdGhlIHNhbXBsZXMKKiBjb2xvcnM6ICAgICAgICAgICAgICAgICAgQ29sb3JzIGNob3NlbiBmb3IgZ3JhcGhzIGFuZCBzdWNoCiogbmFtZXM6ICAgICAgICAgICAgICAgICAgIEJyaW5naW5nIHRvZ2V0aGVyIHRoZSBjb25kaXRpb24vYmF0Y2gKKiBmaWx0ZXJlZDogICAgICAgICAgICAgICAgbG93LWNvdW50IGZpbHRlcmluZyBzdGF0dXMgb2YgdGhlIGNvdW50cwoqIHRyYW5zZm9ybTogICAgICAgICAgICAgICB0cmFuc2Zvcm1hdGlvbiBhcHBsaWVkIHRvIHRoZSBjb3VudHMKKiBub3JtOiAgICAgICAgICAgICAgICAgICAgbm9ybWFsaXphdGlvbiBhcHBsaWVkIHRvIHRoZSBjb3VudHMKKiBjb252ZXJ0OiAgICAgICAgICAgICAgICAgY3BtL3Jwa20vZXRjIGFwcGxpZWQgdG8gdGhlIGRhdGEKKiBvcmlnaW5hbF9saWJzaXplOiAgICAgICAgdGhlIGxpYnJhcnkgc2l6ZXMgYmVmb3JlIG5vcm1hbGl6YXRpb24KKiBjb2x1bW5zOiAgICAgICAgICAgICAgICAgQSBiYWNrdXAgb2YgdGhlIHNhbXBsZSBuYW1lcwoKVGhlIHByaW1hcnkgcmVhc29uIEkgY3JlYXRlZCB0aGUgZXhwdCBvYmplY3Qgd2FzIHRoYXQgSSBkaWQgbm90IGZ1bGx5IHVuZGVyc3RhbmQgaG93IGV4cHJlc3Npb25TZXRzCndvcmtlZC4gIEZyb20gbXkgcGVyc3BlY3RpdmUsIHRoZSBkb2N1bWVudGF0aW9uIHdhcyByYXRoZXIgb3BhcXVlIGFuZCB0aGVyZWZvcmUgSSBkZWNpZGVkIHRvIGNyZWF0ZQphIHNpbXBsZXIgdmVyc2lvbiBvZiB0aGUgZXhwcmVzc2lvbnNldC4gIEFzIEkgbGVhcm5lZCBob3cgdG8gbWFuaXB1bGF0ZSBTNCBjbGFzc2VzIG1vcmUgZWZmaWNpZW50bHksCkkgZ3JhZHVhbGx5IGJlZ2FuIHRvIHJlYWxpemUgdGhhdCB0aGV5IGFyZSBub3Qgc28gYmFkLiAgTm9uZXRoZWxlc3MsIEkgZm91bmQgaXQgbmljZSB0byBiZSBhYmxlIHRvCmtlZXAgc29tZSBleHRyYSBzdGF0ZSBpbmZvcm1hdGlvbiB3aXRoIG15IGV4cHJlc3Npb25zZXRzLCBhbG9uZyB3aXRoIGEgYmFja3VwIGNvcHkgaW4gY2FzZSBvZgptaXN0YWtlcywgYW5kIHNvbWUgZWFzaWVyIHdheXMgdG8gZXh0cmFjdCBpbmZvcm1hdGlvbiBsaWtlIGNvbmRpdGlvbnMvYmF0Y2hlcy9jb2xvcnMvZXRjLiAgVGh1cyBteQoqZXhwdCgpIGZ1bmN0aW9ucyBncmV3IGludG8gd3JhcHBlcnMgdG8gY2FsbCB0aGUgbmF0aXZlIGZ1bmN0aW9ucyBmb3IgbWFuaXB1bGF0aW5nIGV4cHJlc3Npb25zZXRzLgoKIyBSYXcgbWV0cmljcwoKV2l0aCB0aGUgYWJvdmUgaW4gbWluZCwgb25jZSB3ZSBoYXZlIHRoZSB2YXJpb3VzIG1ldGFkYXRhIGFuZCBjb3VudCBkYXRhIGxvYWRlZCBpbnRvIG1lbW9yeSwgYSBtb3N0CmNvbW1vbiBuZXh0IHN0ZXAgaXMgZXhhbWluZSB0aGUgZGF0YSBiZWZvcmUgbW9sZXN0aW5nIGl0OgoKYGBge3IgZ3JhcGhfb3JpZ2luYWwsIGZpZy5zaG93ID0gImhpZGUifQpyYXdfbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKGV4cHQsIHFxID0gVFJVRSwgY2lzID0gTlVMTCkpCmBgYAoKVGhlIGZ1bmN0aW9uIGdyYXBoX21ldHJpY3MoKSBwZXJmb3JtcyBhbGwgb2YgdGhlIGxpa2VseSBwbG90cyBvbmUgbWlnaHQgd2FudC4gIFNvbWUgb2Ygd2hpY2ggYXJlIG5vdApyZWFsbHkgYXBwcm9wcmlhdGUgZm9yIG5vbi1ub3JtYWxpemVkIGRhdGEgdW5sZXNzIGl0IGlzIGluY3JlZGlibHkgd2VsbCBiZWhhdmVkIChhZnRlciAzMCB5ZWFycywgSQpzdGlsbCB3YW50IHRvIHNwZWxsIGJlaGF2ZWQgJ2JlaGFpdmVkJywgd2h5IGlzIHRoYXQ/KS4KCmBgYHtyIHNob3dfb3JpZ2luYWxfcGxvdHN9CiMjIFZpZXcgYSByYXcgbGlicmFyeSBzaXplIHBsb3QKcmF3X21ldHJpY3MkbGlic2l6ZQojIyBPciBib3hwbG90IHRvIHNlZSB0aGUgZGF0YSBkaXN0cmlidXRpb24KcmF3X21ldHJpY3MkYm94cGxvdAojIyBUaGUgd2FybmluZyBpcyBiZWNhdXNlIGl0IGF1dG9tYXRpY2FsbHkgdXNlcyBhIGxvZyBzY2FsZSBhbmQgdGhlcmUgYXJlIHNvbWUgMCBjb3VudCBnZW5lcy4KIyMgUGVyaGFwcyB5b3UgcHJlZmVyIGRlbnNpdHkgcGxvdHMKcmF3X21ldHJpY3MkZGVuc2l0eQojIyBxdWFudGlsZS9xdWFudGlsZSBwbG90cyBjb21wYXJlZCB0byB0aGUgbWVkaWFuIG9mIGFsbCBzYW1wbGVzCnJhd19tZXRyaWNzJHFxcmF0CnJhd19tZXRyaWNzJHRzbmVfcGxvdAojIyBIZXJlIHdlIGNhbiBzZWUgc29tZSBzYW1wbGVzIGFyZSBkaWZmZXJlbnRseSAnc2hhcGVkJyBjb21wYXJlZCB0byB0aGUgbWVkaWFuIHRoYW4gb3RoZXJzCiMjIFRoZXJlIGFyZSBvdGhlciBwbG90cyBvbmUgbWF5IHZpZXcsIGJ1dCB0aGlzIGRhdGEgc2V0IGlzIGEgYml0IHRvbyBjcm93ZGVkIGFzIGlzLgojIyBUaGUgZm9sbG93aW5nIHN1bW1hcnkgc2hvd3MgdGhlIG90aGVyIGF2YWlsYWJsZSBwbG90czoKc3VtbWFyeShyYXdfbWV0cmljcykKYGBgCgpUaGUgcGxvdHMgYXJlIGFsbCBnZW5lcmF0ZWQgYnkgY2FsbGluZyBwbG90X3NvbWV0aGluZygpIHdoZXJlIHRoZSBzb21ldGhpbmdzIGFyZToKCiogbm9uemVybzogc2NhdHRlciBwbG90IG9mIG51bWJlciBvZiBub24temVybyBnZW5lcyB3aXRoIHJlc3BlY3QgdG8gQ1BNIHBzZXVkb2NvdW50cy4KKiBsaWJzaXplOiBiYXIgcGxvdCBvZiB0aGUgKHBzZXVkbykgY291bnRzIGluIGFubm90YXRlZCBmZWF0dXJlcyBvYnNlcnZlZCBieSBzYW1wbGUuCiogYm94cGxvdDogYm94cGxvdCBkZXNjcmliaW5nIHRoZSBkaXN0cmlidXRpb24gb2YgY291bnRzIHdpdGggcmVzcGVjdCB0byBmZWF0dXJlcyBieSBzYW1wbGUuCiogY29yaGVhdDogY29ycmVsYXRpb24gaGVhdCBtYXAgZGVzY3JpYmluZyB0aGUgcmVsYXRpdmUgc2ltaWxhcml0aWVzIGFtb25nIHNhbXBsZXMuCiogc21jOiAnc3RhbmRhcmQgbWVkaWFuIGNvcnJlbGF0aW9uJywgdGFrZSB0aGUgcGFpcndpc2UgY29ycmVsYXRpb25zIG9mIGFsbCBzYW1wbGVzLCBjYWxjdWxhdGUgdGhlCiAgbWVkaWFucywgYW5kIGxvb2sgZm9yIGEgc2luZ2xlIHNhbXBsZSB3aXRoIHNpZ25pZmljYW50bHkgbG93ZXIgY29ycmVsYXRpb25zIChmYWxsaW5nIGJlbG93IHRoZSByZWQKICBsaW5lKS4KKiBkaXNoZWF0OiBkaXN0YW5jZSBoZWF0IG1hcC4gIEFzIGFib3ZlIGJ1dCB3aXRoIGEgZGlzdGFuY2UgbWV0cmljLgoqIHNtZDogJ3N0YW5kYXJkIG1lZGlhbiBkaXN0YW5jZScsIGFzIGFib3ZlIGJ1dCB1c2luZyB0aGUgZGlzdGFuY2UgbWV0cmljcyBhbmQgYSBsaW5lIGFib3ZlLgoqIHBjYXBsb3Q6IHBsb3RfcGNhKCkgZ2l2ZXMgbWFueSBvdXRwdXRzLCBpbmNsdWRpbmcgYSBQQ0EgcGxvdCBvZiB0aGUgZmlyc3QgMiBwcmluY2lwYWwgY29tcG9uZW50cy4KKiBwY2F0YWJsZTogYW5kIGEgdGFibGUgZGVzY3JpYmluZyB0aGUgc2FtZSBtZXRhZGF0YS4KKiBwY2FyZXM6IGFsb25nIHdpdGggdGhlIHRhYmxlIG9mIHZhcmlhbmNlLCBjdW11bGF0aXZlIHZhcmlhbmNlLCBjb25kaXRpb24vYmF0Y2ggcl4yLgoqIHBjYXZhcjogYW5kIHRoZSB2YXJpYW5jZSBieSBjb21wb25lbnQgb2JzZXJ2ZWQuCiogZGVuc2l0eTogcHJldHR5IG11Y2ggdGhlIGV4YWN0IHNhbWUgdGhpbmcgYXMgdGhlIGJveHBsb3QgYWJvdmUsIGJ1dCBhcyBhIGRlbnNpdHkgcGxvdC4KKiBsZWdlbmQ6IGEgY29udmVuaWVudCBsZWdlbmQgZm9yIGZpZ3VyZXMgYW5kIHN1Y2guCiogcXFsb2c6IHFxLXBsb3RzIG9mIGVhY2ggc2FtcGxlIHZzLiB0aGUgbWVkaWFuIG9uIHRoZSBsb2cgc2NhbGUuCiogcXFyYXQ6IHFxLXBsb3RzIG9mIGVhY2ggc2FtcGxlIHZzLiB0aGUgbWVkaWFuIGFzIHJhdGlvcy4KKiBtYTogYmxhbmQtYWx0bWFuIHBsb3RzIG9mIGVhY2ggc2FtcGxlIG9mIE0obG9nIGFidW5kYW5jZSkgd2l0aCByZXNwZWN0IHRvIEEobWVhbiBhdmVyYWdlKS4KCiMgU3Vic2V0dGluZyBkYXRhCgpPbiB0aGUgb3RoZXIgaGFuZCwgd2UgbWlnaHQgdGFrZSBhIHN1YnNldCBvZiB0aGUgZGF0YSB0bwpmb2N1cyBvbiB0aGUgbGF0ZS1sb2cgdnMuIGVhcmx5LWxvZyBzYW1wbGVzLgoKVGhlIGV4cHRfc3Vic2V0KCkgZnVuY3Rpb24gYWxsb3dzIG9uZSB0byBwdWxsIG1hdGVyaWFsIGZyb20gdGhlCmV4cGVyaW1lbnRhbCBkZXNpZ24uCgpPbmNlIHdlIGhhdmUgYSBzbWFsbGVyIGRhdGEgc2V0LCB3ZSBjYW4gbW9yZSBlYXNpbHkgdXNlIFBDQSB0byBzZWUgaG93CnRoZSBzYW1wbGUgc2VwYXJhdGUuCgpgYGB7ciBzdWJzZXRfZGF0YSwgZmlnLnNob3c9J2hpZGUnfQpoZWFkKGV4cHQkZGVzaWduKQojIyBlbHQgc3RhbmRzIGZvcjogImVhcmx5L2xhdGUgaW4gdGh5IgpiYXRjaF9hIDwtIHN1YnNldF9leHB0KGV4cHQsIHN1YnNldCA9ICJiYXRjaD09J2EnIikKYmF0Y2hfYiA8LSBzdWJzZXRfZXhwdChleHB0LCBzdWJzZXQgPSAiYmF0Y2g9PSdiJyIpCgphX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhiYXRjaF9hLCBjaXMgPSBOVUxMKSkKYl9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3MoYmF0Y2hfYiwgY2lzID0gTlVMTCkpCmBgYAoKUENBIHBsb3RzIG9mIHRoZXNlIHN1YnNldHRlZCBkYXRhIGFyZSBub3QgbGlrZWx5IHRvIGJlIHZlcnkgaW50ZXJlc3RpbmcsIGFzIGl0CmhhcyBiZWVuIHJlZHVjZWQgdG8gYSBzaW5nbGUgc2FtcGxlIHBlciBjb25kaXRpb24sIGJ1dCB3ZSBjYW4gbG9vayBhdCB0aGVtCmFueWhvdy4KCmBgYHtyIHN1YnNldF9zaG93X3Bsb3RzfQphX21ldHJpY3MkcGNfcGxvdApiX21ldHJpY3MkcGNfcGxvdAphX21ldHJpY3MkdHNuZV9wbG90CmJfbWV0cmljcyR0c25lX3Bsb3QKYGBgCgojIE5vcm1hbGl6aW5nIGRhdGEKCkl0IGlzIHByZXR0eSBvYnZpb3VzIHRoYXQgdGhlIHJhdyBkYXRhIGlzIGEgYml0IGp1bWJsZWQgYWNjb3JkaW5nIHRvIFBDQS4gIFRoaXMgaXMgbm90IHBhcmljdWxhcmx5CnN1cHJpc2luZyBzaW5jZSB3ZSBkaWRuJ3Qgbm9ybWFsaXplIGl0IGF0IGFsbC4gIFRoZXJlZm9yZSwgaW4gdGhpcyBibG9jayBJIHdpbGwgbm9ybWFsaXplIGl0IGEgZmV3CndheXMgYW5kIGZvbGxvdyB1cCB3aXRoIHNvbWUgdmlzdWFsaXphdGlvbnMgb2Ygc2hvd2luZyBob3cgdGhlIGFwcGFyZW50IHJlbGF0aW9uc2hpcHMgY2hhbmdlIGluIHRoZQpkYXRhLgoKYGBge3Igbm9ybWFsaXplX3N1YnNldCwgZmlnLnNob3cgPSAiaGlkZSJ9CiMjIGRvaW5nIG5vdGhpbmcgdG8gdGhlIGRhdGEgZXhjZXB0IGxvZzIgdHJhbnNmb3JtaW5nIGl0IGhhcyBhIHN1cnByaXNpbmdseSBsYXJnZSBlZmZlY3QKbm9ybV90ZXN0IDwtIG5vcm1hbGl6ZV9leHB0KGV4cHQsIHRyYW5zZm9ybSA9ICJsb2cyIikKbDJfbWV0cmljcyA8LSBzbShncmFwaF9tZXRyaWNzKG5vcm1fdGVzdCwgY2lzID0gTlVMTCkpCiMjIGEgcXVhbnRpbGUgbm9ybWFsaXphdGlvbiBhbG9uZSBhZmZlY3Qgc29tZSwgYnV0IG5vdCBhbGwgb2YgdGhlIGRhdGEKbm9ybV90ZXN0IDwtIHNtKG5vcm1hbGl6ZV9leHB0KGV4cHQsIG5vcm0gPSAicXVhbnQiKSkKcV9tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3Mobm9ybV90ZXN0LCBjaXMgPSBOVUxMKSkgICMjIHEgZm9yIHF1YW50LCB3aG8gcXVhZmZlZCBuaWdodHNoYWRlLgojIyBjcG0gYWxvbmUgYnJpbmdzIG91dCBzb21lIHNhbXBsZXMsIHRvbwpub3JtX3Rlc3QgPC0gc20obm9ybWFsaXplX2V4cHQoZXhwdCwgY29udmVydCA9ICJjcG0iKSkKY19tZXRyaWNzIDwtIHNtKGdyYXBoX21ldHJpY3Mobm9ybV90ZXN0LCBjaXMgPSBOVUxMKSkgICMjIGMgZm9yIGNwbSwgd2hvIGNvdWxkIG5vdCBzZWUgdGhlIHRyYWluLgojIyBsb3cgY291bnQgZmlsdGVyaW5nIGhhcyBzb21lIGVmZmVjdCwgdG9vCm5vcm1fdGVzdCA8LSBzbShub3JtYWxpemVfZXhwdChleHB0LCBmaWx0ZXIgPSAicG9mYSIpKQpmX21ldHJpY3MgPC0gc20oZ3JhcGhfbWV0cmljcyhub3JtX3Rlc3QsIGNpcyA9IE5VTEwpKSAgIyMgZiBmb3IgZmlsdGVyLCB3aG8gd2FzIGhpdCB3aXRoIGEgc3BhZGUuCiMjIGhvdyBhYm91dCBpZiB3ZSBtaXggYW5kIG1hdGNoIG1ldGhvZHM/Cm5vcm1fdGVzdCA8LSBzbShub3JtYWxpemVfZXhwdChleHB0LCB0cmFuc2Zvcm0gPSAibG9nMiIsIGNvbnZlcnQgPSAiY3BtIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5vcm0gPSAicXVhbnQiLCBiYXRjaCA9ICJjb21iYXRfc2NhbGUiLCBmaWx0ZXIgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmF0Y2hfc3RlcCA9IDQsIGxvd190b196ZXJvID0gVFJVRSkpCiMjIFNvbWUgbWV0cmljcyBhcmUgbm90IHZlcnkgdXNlZnVsIG9uIChlc3BlY2lhbGx5IHF1YW50aWxlKSBub3JtYWxpemVkIGRhdGEKbm9ybV9ncmFwaHMgPC0gc20oZ3JhcGhfbWV0cmljcyhub3JtX3Rlc3QsIGNpcyA9IE5VTEwpKQpgYGAKCk5vdyBsZXRzIHNlZSBzb21lIG9mIHRoZSByZXN1bHRpbmcgbWV0cmljcywgaW4gdGhpcyBjYXNlIEkgd2lsbCBqdXN0IGNvbXBhcmUgc29tZSBwY2EgcGxvdHMsIGFzIHRoZXkKYXJlIGdvb2QgYXQgZm9vbGluZyBvdXIgc2lsbHkgdmlzdWFsIGJyYWlucyBpbnRvIHNlZWluZyBwYXR0ZXJucy4KCmBgYHtyIHZpZXdfbWV0cmljc30KbDJfbWV0cmljcyRwY19wbG90CiMjIEFsc28gdmlld2FibGUgd2l0aCBwbG90X3BjYSgpJHBsb3QKIyMgUENBIHBsb3RzIHNlZW0gKHRvIG1lKSB0byBwcmVmZXIgbG9nMiBzY2FsZSBkYXRhLgpxX21ldHJpY3MkcGNfcGxvdAojIyBvbmx5IG5vcm1hbGl6aW5nIG9uIHRoZSBxdWFudGlsZXMgbGVhdmVzIHRoZSBkYXRhIG9wZW4gdG8gc2NhbGUgZWZmZWN0cy4KY19tZXRyaWNzJHBjX3Bsb3QKIyMgYnV0IGNwbSBhbG9uZSBpcyBpbnN1ZmZpY2llbnQKZl9tZXRyaWNzJHBjX3Bsb3QKIyMgb25seSBmaWx0ZXJpbmcgb3V0IGxvdy1jb3VudCBnZW5lcyBpcyBoZWxwZnVsIGFzIHdlbGwKbm9ybV9ncmFwaHMkcGNfcGxvdAojIyBUaGUgZGlmZmVyZW50IGJhdGNoIGVmZmVjdCB0ZXN0aW5nIG1ldGhvZHMgaGF2ZSBhIHByZXR0eSB3aWRlbHkgcmFuZ2luZyBlZmZlY3Qgb24gdGhlIGNsdXN0ZXJpbmcKIyMgcGxheSB3aXRoIHRoZW0gYnkgY2hhbmdpbmcgdGhlIGJhdGNoPSBwYXJhbWV0ZXIgdG86CiMjICJsaW1tYSIsICJzdmEiLCAic3Zhc2VxIiwgImxpbW1hcmVzaWQiLCAicnV2ZyIsICJjb21iYXQiLCBjb21iYXRtb2QiCmtuaXRyOjprYWJsZShub3JtX2dyYXBocyRwY19zdW1tYXJ5KQojIyBUaHVzIHdlIHNlZSBhIGRyYW1hdGljIGRlY3JlYXNlIGluIHZhcmlhbmNlIGFjY291bnRlZCBmb3IKIyMgYnkgYmF0Y2ggYWZ0ZXIgYXBwbHlpbmcgbGltbWEncyAncmVtb3ZlYmF0Y2hlZmZlY3QnCiMjIChzZWUgYmF0Y2guUjIgaGVyZSB2cy4gYWJvdmUpCm5vcm1fZ3JhcGhzJHNtYwpub3JtX2dyYXBocyRkaXNoZWF0ICAjIyBzdmFzZXEncyBiYXRjaCBjb3JyZWN0aW9uIHNlZW1zIHRvIGRyYXcgb3V0IHRoZSBzaWduYWwgcXVpdGUgbmljZWx5LgojIyBJdCBpcyB3b3J0aCBub3RpbmcgdGhhdCB0aGUgd3QsIGVhcmx5IGxvZywgdGh5LCByZXBsaWNhdGUgYyBzYW1wbGVzIGFyZSBzdGlsbCBhIGJpdCB3ZWlyZC4Kbm9ybV9ncmFwaHMkdHNuZV9wbG90CmBgYAoKIyBQZXJmb3JtaW5nIERFIGFuYWx5c2VzCgpUaGlzIGlzIGEgcmVsYXRpdmVseSBzbWFsbCBkYXRhIHNldCwgc28gcGVyZm9ybWluZyBzb21lIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2VzIHJlYWxseQpzaG91bGQgbm90IHRha2UgbG9uZyBhdCBhbGwuCgpXaGVuIHBlcmZvcm1pbmcgdGhlc2UgYW5hbHlzZXMgd2l0aCBocGdsdG9vbHMsIGl0IHdpbGwgYXR0ZW1wdCB0byBwZXJmb3JtIHNpbWlsYXIgYW5hbHlzZXMgd2l0aApsaW1tYSwgZWRnZVIsIGFuZCBERVNlcTIgdmlhIHRoZSBhbGxfcGFpcndpc2UoKSBmdW5jdGlvbi4gIFRoZSBtb3N0IGxpa2VseSBhcmd1bWVudCBpcyAnbW9kZWxfYmF0Y2gnCndoaWNoIG1heSBiZSB1c2VkIHRvIGV4cGxpY2l0bHkgaW5jbHVkZS9leGNsdWRlIGEgYmF0Y2ggZmFjdG9yIGluIHRoZSBtb2RlbCwgb3IgYXNrIGl0IHRvIGF0dGVtcHQKaW5jbHVkaW5nIGJhdGNoIGZhY3RvcnMgZnJvbSBzdmEvcnV2L2V0Yy4gIEJ5IGRlZmF1bHQgaXQgd2lsbCBhdHRlbXB0IHRvIGluY2x1ZGUgYSBjb2x1bW4gZnJvbSB0aGUKZXhwZXJpbWVudGFsIGRlc2lnbiBuYW1lZCAnYmF0Y2gnLgoKYGBge3IgZGVfdGVzdH0Kc3B5b2dlbmVzX2RlIDwtIHNtKGFsbF9wYWlyd2lzZShleHB0KSkKIyMgRXZlbiB0aGUgbG93ZXN0IGNvcnJlbGF0aW9ucyBhcmUgcXVpdGUgaGlnaC4KYGBgCgpUaGUgcmVzdWx0IG9mIGFsbF9wYWlyd2lzZSgpIGlzIGEgbGlzdCBvZiB0aGUgcmVzdWx0cyBmcm9tIGxpbW1hLCBlZGdlciwgYW5kIGRlc2VxLiAgSW4gYWRkaXRpb24sIEkKaW1wbGVtZW50ZWQgYSB2ZXJ5IHNpbXBsaXN0aWMsIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZ1bmN0aW9uIG5hbWVkICdiYXNpYygpJy4gIEl0IGFsc28gcHJvdmlkZXMKc29tZSBzaW1wbGUgbWVhc3VyZW1lbnRzIG9mIGhvdyB3ZWxsIHRoZSB2YXJpb3VzIGFuYWx5c2VzIGFncmVlIChlcmdvIHRoZSBibGFjayBhbmQgd2hpdGUgaGVhdG1hcCkuCgpXb3JraW5nIHdpdGggdGhlc2Ugc2VwYXJhdGUgdGFibGVzIGNhbiBiZSBtb3JlIHRoYW4gYSBsaXR0bGUgYW5ub3lpbmcsIGNvbWJpbmVfZGVfdGFibGVzKCkgYXR0ZW1wdHMKdG8gc2ltcGxpZnkgdGhpcy4gIEl0IHdpbGwgYnJpbmcgdG9nZXRoZXIgdGhlIHZhcmlvdXMgdGFibGVzLCBhbmQgaWYgYXNrZWQgYXR0ZW1wdCB0byBicmluZyB0aGVtCnRvZ2V0aGVyIGludG8gYSBwcmV0dHktaWZpZWQgZXhjZWwgd29ya2Jvb2suCgphbGxfcGFpcndpc2UoKSBhcmJpdHJhcmlseSBwZXJmb3JtcyBhbGwgcG9zc2libGUgcGFpcndpc2UgY29tcGFyaXNvbnMuICBUaGlzIGlzIG5vdCBuZWNlc3NhcmlseSB3aGF0Cm9uZSBhY3R1YWxseSB3aXNoZXMgdG8gc2VlLiAgVGhlcmVmb3JlLCB0aGUgYXJndW1lbnQga2VlcGVycyB0YWtlcyBhIGxpc3Qgb2YgY29udHJhc3RzOgoKYGBge3Iga2VlcGVyX2V4YW1wbGV9Cm15X2tlZXBlcnMgPC0gbGlzdCgKICAjIyBuYW1lICAgID0gICBudW1lcmF0b3IgLyBkZW5vbWluYXRvcgogICJ3dF9tZWRpYSIgPSBjKCJ3dF9sbF9jZiIsICJ3dF9sbF9jZyIpLAogICJtZ2FfbWVkaWEiID0gYygibWdhX2xsX2NmIiwgIm1nYV9sbF9jZyIpKQpgYGAKCkluIHRoZSBhYm92ZSBleGFtcGxlLCBpZiB0aGUga2VlcGVycyBhcmd1bWVudCB0byBjb21iaW5lX2RlX3RhYmxlcygpIGlzIGdpdmVuIGFzIG15X2tlZXBlcnMsIHRoZW4KdGhlIHJlc3VsdGluZyB0YWJsZSB3aWxsIG5vdCBoYXZlIHRoZSBzZXQgb2YgNiBwb3NzaWJsZSBjb21wYXJpc29ucywgYnV0IGluc3RlYWQgd2lsbCBvbmx5IGhhdmUgMgp0YWJsZXMgbmFtZWQgJ3d0X21lZGlhJyBhbmQgJ21nYV9tZWRpYScsIHdoaWNoIGlmIHByaW50ZWQgdG8gZXhjZWwgd2lsbCBiZSBzaGVldHMgbmFtZWQgYWNjb3JkaW5nbHkuCgpgYGB7ciBjb21iaW5lX3Rlc3R9CnNweW9nZW5lc190YWJsZXMgPC0gc20oY29tYmluZV9kZV90YWJsZXMoc3B5b2dlbmVzX2RlLCBleGNlbCA9IEZBTFNFKSkKc3VtbWFyeShzcHlvZ2VuZXNfdGFibGVzKQojIyBUcnkgY2hhbmdpbmcgdGhlIHAtYWRqdXN0bWVudApzcHlvZ2VuZXNfdGFibGVzIDwtIHNtKGNvbWJpbmVfZGVfdGFibGVzKHNweW9nZW5lc19kZSwgZXhjZWwgPSBGQUxTRSwgcGFkal90eXBlID0gIkJIIikpCmtuaXRyOjprYWJsZShoZWFkKHNweW9nZW5lc190YWJsZXMkZGF0YVtbMV1dKSkKYGBgCgpGaW5hbGx5LCBleHRyYWN0X3NpZ25pZmljYW50X2dlbmVzKCkgbWF5IGNob29zZSAnc2lnbmlmaWNhbnQnIGdlbmVzIGJhc2VkIHVwb24gYSBmZXcgbWV0cmljcwppbmNsdWRpbmcgei1zY29yZSB2cy4gdGhlIGRpc3RyaWJ1dGlvbiBvZiBsb2dGQzsgYSBsb2dGQyBjdXRvZmYsIChhanVzdGVkKXAtdmFsdWUgY3V0b2ZmLCBhbmQvb3IKdG9wL2JvdHRvbSBuIGdlbmVzLgoKYGBge3Igc2lnX2dlbmVzX3Rlc3QsIGZpZy5zaG93ID0gImhpZGUifQpzcHlvZ2VuZXNfc2lnIDwtIHNtKGV4dHJhY3Rfc2lnbmlmaWNhbnRfZ2VuZXMoc3B5b2dlbmVzX3RhYmxlcywgZXhjZWwgPSBGQUxTRSkpCmtuaXRyOjprYWJsZShoZWFkKHNweW9nZW5lc19zaWckbGltbWEkdXBzW1sxXV0pKQpgYGAKCiMgTWFrZSBwcmV0dHkgY2lyY29zIGdyYXBocwoKU2luY2UgbW9zdCBvZiBteSBjaXJjb3MgZ3JhcGhzIGFyZSBmb3IgcHlvZ2VuZXMsIGl0IGlzIGxpa2VseSB0aGF0IHRoZSBkZWZhdWx0cyBhcmUgYXBwcm9wcmlhdGUgZm9yCnRoaXMgcGFydGljdWxhciBvcmdhbmlzbS4KCk11Y2goYWxsKSBvZiB0aGUgZm9sbG93aW5nIGlzIHRha2VuIGZyb20gdGhlIG1hdGVyaWFsIGluIHRlc3RzL3Rlc3R0aGF0L3Rlc3RfNzBtZ2EuUgoKYGBge3IgY2lyY29zfQojI21pY3JvYmVfaWRzIDwtIGFzLmNoYXJhY3RlcihzbShnZXRfbWljcm9iZXNvbmxpbmVfaWRzKCJweW9nZW5lcyBNR0FTNTAwNSIpKSkKIyMgQSBjYXZlYXQhICBUaGUgbmV3IHZlcnNpb24gb2YgbWljcm9iZXNvbmxpbmUgY2hhbmdlZCB0aGUgSURzIHNvIHRoYXQgdGhleSBubyBsb25nZXIKIyMgbWF0Y2ggbXkgb2xkIHJuYXNlcSBhbmFseXNpcyEhICBUaHVzIEkgcHV0IG15IG9sZCBnZmYgZmlsZSB1c2VkIGZvciBtYXBwaW5nIGludG8gaW5zdC8KIyMgYW5kIHdpbGwgbG9hZCB0aGUgYW5ub3RhdGlvbiBkYXRhIGZyb20gdGhhdDsgYnV0IEkgd2lsbCB1c2UgdGhpcyBkYXRhIHRvIGdhdGhlcgojIyB0aGUgQ09HIGluZm9ybWF0aW9uLgptZ2FzX2dmZl9kZiA8LSBzbShsb2FkX2dmZl9hbm5vdGF0aW9ucyhnZmYgPSBzeXN0ZW0uZmlsZSgic2hhcmUvZ2FzLmdmZiIsIHBhY2thZ2UgPSAiaHBnbHRvb2xzIikpKQptZ2FzX2dmZl9kZiA8LSBtZ2FzX2dmZl9kZlstMSwgXQptZ2FzX2dmZl9kZltbInN5c05hbWUiXV0gPC0gZ3N1YihwYXR0ZXJuID0gIlNweV8iLCByZXBsYWNlbWVudCA9ICJTcHkiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB4ID0gbWdhc19nZmZfZGZbWyJsb2N1c190YWciXV0pCnJvd25hbWVzKG1nYXNfZ2ZmX2RmKSA8LSBtYWtlLm5hbWVzKG1nYXNfZ2ZmX2RmW1sic3lzTmFtZSJdXSwgdW5pcXVlID0gVFJVRSkKCm1nYXNfbWljcm9iZXNfZGYgPC0gc20obG9hZF9taWNyb2Jlc29ubGluZV9hbm5vdGF0aW9ucyhpZCA9IDI5MzY1MykpCm1nYXNfbWljcm9iZXNfZGYkc3lzTmFtZSA8LSBnc3ViKHBhdHRlcm4gPSAiU3B5XyIsIHJlcGxhY2VtZW50ID0gIlNweSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHggPSBtZ2FzX21pY3JvYmVzX2RmJHN5c05hbWUpCnJvd25hbWVzKG1nYXNfbWljcm9iZXNfZGYpIDwtIG1ha2UubmFtZXMobWdhc19taWNyb2Jlc19kZiRzeXNOYW1lLCB1bmlxdWUgPSBUUlVFKQoKbWdhc19kZiA8LSBtZXJnZSh4ID0gbWdhc19nZmZfZGYsIHkgPSBtZ2FzX21pY3JvYmVzX2RmLCBieSA9ICJyb3cubmFtZXMiKQpyb3duYW1lcyhtZ2FzX2RmKSA8LSBtZ2FzX2RmW1siUm93Lm5hbWVzIl1dCm1nYXNfZGYgPC0gbWdhc19kZlssIC0xXQpjb2xuYW1lcyhtZ2FzX2RmKSA8LSBjKCJzZXFuYW1lcyIsICJzdGFydCIsICJlbmQiLCAid2lkdGgiLCAic3RyYW5kIiwgInNvdXJjZSIsICJ0eXBlIiwKICAgICAgICAgICAgICAgICAgICAgICAic2NvcmUiLCAicGhhc2UiLCAiSUQiLCAiRGJ4cmVmIiwgIklzX2NpcmN1bGFyIiwgImdia2V5IiwgImdlbm9tZSIsCiAgICAgICAgICAgICAgICAgICAgICAgIm1vbF90eXBlIiwgInN0cmFpbiIsICJOYW1lIiwgIk5vdGUiLCAiZ2VuZSIsICJsb2N1c190YWciLAogICAgICAgICAgICAgICAgICAgICAgICJQYXJlbnQiLCAicHJvZHVjdCIsICJwcm90ZWluX2lkIiwgInRyYW5zbF90YWJsZSIsICJnZW5lX3N5bm9ueW0iLAogICAgICAgICAgICAgICAgICAgICAgICJzeXNOYW1lX2FnYWluIiwgImxvY3VzSWQiLCAiYWNjZXNzaW9uIiwgIkdJIiwgInNjYWZmb2xkSWQiLAogICAgICAgICAgICAgICAgICAgICAgICJzdGFydF9hZ2FpbiIsICJzdG9wIiwgInN0cmFuZF9hZ2FpbiIsICJzeXNOYW1lX2FnYWluIiwgIm5hbWUiLAogICAgICAgICAgICAgICAgICAgICAgICJkZXNjIiwgIkNPRyIsICJDT0dGdW4iLCAiQ09HRGVzYyIsICJUSUdSRmFtIiwgIlRJR1JSb2xlcyIsCiAgICAgICAgICAgICAgICAgICAgICAgIkdPIiwgIkVDIiwgIkVDRGVzYyIpCgojIyBGaXJzdCBtYWtlIGEgdGVtcGxhdGUgY29uZmlndXJhdGlvbgpjaXJjb3NfdGVzdCA8LSBjaXJjb3NfcHJlZml4KGFubm90YXRpb24gPSBtZ2FzX2RmKQojIyBGaWxsIGl0IGluIHdpdGggdGhlIGRhdGEgZm9yIHMucHlvZ2VuZXMKbGVuZ3RocyA8LSAxODM4NjAwCm5hbWVzKGxlbmd0aHMpIDwtICJOQ18wMDcyOTciCgpjaXJjb3Nfa2FyeSA8LSBjaXJjb3Nfa2FyeW90eXBlKGNmZyA9IGNpcmNvc190ZXN0LCBsZW5ndGhzID0gbGVuZ3RocykKIyMgRmlsbCBpbiB0aGUgZ2VuZSBjYXRlZ29yeSBhbm5vdGF0aW9ucyBieSBnZW5lLXN0cmFuZApjaXJjb3NfcGx1cyA8LSBjaXJjb3NfcGx1c19taW51cyhjZmcgPSBjaXJjb3NfdGVzdCkKY2lyY29zX2xpbW1hX2hpc3QgPC0gY2lyY29zX2hpc3QoY2ZnID0gY2lyY29zX3Rlc3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRmID0gc3B5b2dlbmVzX2RlJGxpbW1hJGFsbF90YWJsZXNbWzFdXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmFzZW5hbWUgPSAibGltbWEiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xuYW1lID0gImxvZ0ZDIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb3V0ZXIgPSBjaXJjb3NfcGx1cykKY2lyY29zX2Rlc2VxX2hpc3QgPC0gY2lyY29zX2hpc3QoY2ZnID0gY2lyY29zX3Rlc3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRmID0gc3B5b2dlbmVzX2RlJGRlc2VxJGFsbF90YWJsZXNbWzFdXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmFzZW5hbWUgPSAiZGVzZXEiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xuYW1lID0gImxvZ0ZDIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb3V0ZXIgPSBjaXJjb3NfbGltbWFfaGlzdCkKY2lyY29zX2VkZ2VyX2hpc3QgPC0gY2lyY29zX2hpc3QoY2ZnID0gY2lyY29zX3Rlc3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRmID0gc3B5b2dlbmVzX2RlJGVkZ2VyJGFsbF90YWJsZXNbWzFdXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmFzZW5hbWUgPSAiZWRnZXIiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xuYW1lID0gImxvZ0ZDIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb3V0ZXIgPSBjaXJjb3NfZGVzZXFfaGlzdCkKY2lyY29zX3N1ZmZpeChjZmcgPSBjaXJjb3NfdGVzdCkKY2lyY29zX21hZGUgPC0gc20oY2lyY29zX21ha2UoY2ZnID0gY2lyY29zX3Rlc3QsIHRhcmdldCA9ICJtZ2FzIikpCmdldHdkKCkKYGBgCgohW2NpcmNvcyByZXN1bHRdKGNpcmNvcy9tZ2FzLnN2ZykKCkdlbm9wbG90UiBpcyBuZXcgdG8gbWUsIGJ1dCBpdCBzZWVtcyB0byB3b3JrPwoKYGBge3IgZ2Vub3Bsb3R9Cmdlbm9wbG90X2Nocm9tb3NvbWUoKQpgYGAKCiMgQ2hhbmdlIGNvbmRpdGlvbnMKClBlcmhhcHMgaW5zdGVhZCBvZiBsb29raW5nIGF0IHN1YnNldHMgb2YgdGhlIGRhdGEsIHdlIG1heSB3YW50IHRvIGNvbnNpZGVyIHd0L21nYS4KSWYgc28sIHdlIGNhbiBqdXN0IGNoYW5nZSB0aGUgY29uZGl0aW9uIHRvIGFueSBjb2x1bW4gaW4gdGhlIGRlc2lnbiBtYXRyaXguCgpgYGB7ciB3dF9tZ2EsIGZpZy5zaG93ID0gImhpZGUifQp3dF9tZ2FfZXhwdCA8LSBzZXRfZXhwdF9jb25kaXRpb25zKGV4cHQgPSBleHB0LCBmYWN0ID0gInR5cGUiKQp3dF9tZ2FfcGxvdHMgPC0gc20oZ3JhcGhfbWV0cmljcyh3dF9tZ2FfZXhwdCkpCnd0X21nYV9ub3JtIDwtIHNtKG5vcm1hbGl6ZV9leHB0KHd0X21nYV9leHB0LCB0cmFuc2Zvcm0gPSAibG9nMiIsIGNvbnZlcnQgPSAicmF3IiwgZmlsdGVyID0gVFJVRSwgbm9ybSA9ICJxdWFudCIpKQp3dF9tZ2FfbnBsb3RzIDwtIHNtKGdyYXBoX21ldHJpY3Mod3RfbWdhX25vcm0pKQp3dF9tZ2FfZGUgPC0gc20oYWxsX3BhaXJ3aXNlKGlucHV0ID0gd3RfbWdhX2V4cHQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgY29tYmluZWRfZXhjZWwgPSAid3RfbWdhLnhsc3giLAogICAgICAgICAgICAgICAgICAgICAgICAgIHNpZ19leGNlbCA9ICJ3dF9tZ2Ffc2lnLnhsc3giLAogICAgICAgICAgICAgICAgICAgICAgICAgIGFidW5kYW50X2V4Y2VsID0gInd0X21nYV9hYnVuZGFudC54bHN4IikpCmBgYAoKYGBge3Igd3RfbWdhX3Bsb3RzfQp3dF9tZ2FfZGUkY29tYmluZWQkY29tcF9wbG90CiMjIEhvdyB3ZWxsIGRvIHRoZSB2YXJpb3VzIERFIHRvb2xzIGFncmVlIG9uIHRoaXMgZGF0YT8KCnd0X21nYV9wbG90cyR0c25lX3Bsb3QKd3RfbWdhX25wbG90cyRwY19wbG90Cnd0X21nYV9kZSRjb21iaW5lZCRsaW1tYV9wbG90cyRXVF92c19tZ2Ekc2NhdHRlcgp3dF9tZ2FfZGUkY29tYmluZWQkbGltbWFfbWFfcGxvdHMkV1RfdnNfbWdhJHBsb3QKd3RfbWdhX2RlJGNvbWJpbmVkJGxpbW1hX3ZvbF9wbG90cyRXVF92c19tZ2EkcGxvdApgYGAKCmBgYHtyIHN5c2luZm8sIHJlc3VsdHM9J2FzaXMnfQpwYW5kZXI6OnBhbmRlcihzZXNzaW9uSW5mbygpKQpgYGAK